Python Modules

What is module in python:

Let's say you are working from a python interpreter. you spent 30 minutes to define a function, you use it, and quit the interpreter. But suddenly you remember that you still need to do something with that function again. You enter it again but the definition of that function is gone. Oouch, that hurts. Now you again have to spend 30 minutes typing the same function. So python has a way to put this function definition in a file and use it at any time.

Module is the object of type ModuleType. The module is basically a python file (file with the extension .py) that contains the definition and statements (example: class, function, variable, etc.). Modules in Python provides us the flexibility to organize the code logically. We use modules to break-down a large program into small manageable files. We can define functions or classes in a module and reuse them in other modules using the import. Python itself comes with a lot of modules like math, itertools, collection, etc.

Example: Let's create a module. Type the below function in a file and save it as utils.py.

### utils.py 

def multiplication(a: int, b: int)-> int:
    """This function will return the multiplication of two integers"""

    return a * b

So we have defined a function named multiplication inside a module named utils.

How to import modules:

We can import the definitions inside a module to another module using the keyword import. Python provides two ways to import a module as shown below:

import module_name
from module_name import name1, name2...

Example: Let's import the utils module in another module called operations.py

### operations.py

import utils

result = utils.multiplication(2, 4)
print(result)

>>> 8

When we import the utils in the operations module, it does not directly load the objects that are defined inside the utils module into operations module. Instead, it will just load the module name (utils) in the current module namespace. To use the utils defined functions or classes we have to use the dot (.) notation as shown in the above example (utils.multiplication(2, 4)). If you try to use multiplication directly than python will throw an exception as shown below:

>>> import utils
>>> multiplication(2,3)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'multiplication' is not defined

If we want to import a specific function or definition from a module to other modules then we have to use the from module_name import name.

### operations.py

from utils import multiplication

result = multiplication(2, 4)
print(result)

>>>8

Note : Python also allows us to alias our imports using the as keyword as shown below:

import modulename as somename
from modulename import name as somename.

Example:

### operations.py

# import utils as u
from utils import multiplication as mul

# result = u.multiplication (2,4)
result = mul(2, 4)
print(result)

>>>8

Note : We can use the import * or from .. import * to import all objects from a specific module.

Example:

### operations.py

from utils import *

result = multiplication(2, 4)
print(result)

>>>8

Note : We can use the dir() function to find out objects that are defined inside a module.

>>> import utils
>>>dir(utils)
['__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__spec__', 'multiplication']

Here, we can see a list of names (along with multiplication). All names that begin with an underscore are default Python attributes associated with the module. For example __name__ stores the name of the module and __file__ stores the path of the modules where it resides.

>>>import utils
>>>utils.__name__  # name of the module
'utils'
>>>utils.__file__  # module path
'C:\\Users\\user\\Desktop\\python_practice\\utils.py'

How does python import the modules:

Python uses a relatively complex system of how to find and load modules. The sys module has a few properties that define where Python is going to look for modules.

Where does Python look for imports?

>>> import sys
>>> sys.path
['', 'C:\\Users\\user\\Anaconda3\\python37.zip', 'C:\\Users\\user\\Anaconda3\\DLLs', 'C:\\Users\\user\\Anaconda3\\lib', 'C:\\Users\\user\\Anaconda3', 'C:\\Users\\user\\AppData\\Roaming\\Python\\Python37\\site-packages', 'C:\\Users\\user\\Anaconda3\\lib\\site-packages', 'C:\\Users\\user\\Anaconda3\\lib\\site-packages\\win32', 'C:\\Users\\user\\Anaconda3\\lib\\site-packages\\win32\\lib', 'C:\\Users\\user\\Anaconda3\\lib\\site-packages\\Pythonwin']

So when we import a module python will look into the sys.path list. if it does not find the module in one of these paths then it will raise a ModuleNotFoundError.

There are various ways of making sure a directory is always on the Python sys.path list when you run Python, including:

use sys.path.append('module/package directory') before importing.
put the directory into the contents of the PYTHONPATH environment variable.
make the module installable and install it.

At a high level, this is how Python imports a module:

The first place checked during import search is sys.modules. It checks the sys.modules cache to see if the module has already been imported - if yes it simply uses the reference in there, otherwise:
It creates a new module object (types.ModuleType)
It loads the source code from the file
It adds an entry to sys.modules with name as key and the newly created object reference as the value.
It compiles and executes the source code.

Note: When a module is imported, the module code is executed.

What happens when we import a module

import module_name
- loads the entire module(module_name) in memory if it's not already there
- adds a reference to it in sys.modules with a key of module_name
- adds a symbol of the same name in our current namespace referencing the module_name object
from module_name import somename
- loads the entire module (module_name) in memory if it's not already there
- adds a reference to it in sys.modules with a key of module_name
- adds the symbol somename to our current namespace referencing the module_name.somename function
- it does not add the symbol module_name to our current namespace
import module_name as randomename
- loads the entire module (module_name) in memory if it's not already there
- adds a reference to it in sys.modules with a key of module_name
- adds the symbol randomename to our current namespace referencing the module_name object
from module_name import somename as randomname
- loads the entire module (module_name ) in memory if it's not already there
- adds a reference to it in sys.modules with a key of module_name
- adds the symbol randomname to our current namespace referencing the module_name.somename function
- it does not add the symbol module_name to our current namespace
from module_name import *
- loads the entire module (module_name ) in memory if it's not already there
- adds a reference to it in sys.modules with a key of module_name
- adds symbols for all exported symbols in the module_name module directly to our namespace (we can control what is being imported using __all__)
- it does not add the symbol module_name to our current namespace

Conclusions:

In this article, we have covered:

How we can create a Python module
Where the Python interpreter searches for a module
How to import a module
What happens when a module is imported