Python for Scientists – Environments for multiple Operating Systems

Do you want a Conda environment that can be shared between different operating systems? This can come in handy if your project does not contain OS-specific Python packages. Examples where sharing an environment across multiple Operating Systems might be handy are workshops or tutorials. Read on to learn a new way to create and share environments!

Check out this post if you want to create an exact copy of an environment – including every dependency – specific to an operating system.

As a refresher, Conda environments are basically sandboxes. This means you can install any Anaconda-supported version of Python (or R) and associated packages without interfering with other Conda environments. Environments are great for sharing a set of packages for a specific project and for disaster recovery. Awesome!

Topics covered

YAML files
Environment creation
Activate/deactivate environments

Click here for a list of related posts.

Updated: 2020-01-06

YAML? What?

YAML stands for Yet Another Markup Language. It’s easier to create than an XML or JSON document. The format is natural. Have you ever created a text document along the lines of this:

Important thing
- note about important thing
- another note

If so, you already know how to create a basic YAML file.

There are three basic parts to a Conda YAML environment file. The environment name, channels, and dependency list. We have gone over my method of environment naming, but whatever you choose to use remember to be consistent. Consistency is important!

Create a new text file and name it py37_test_environment.yaml (or yml). Remember to change the environment name to reflect your environment. I prefer to use Notepad ++ for editing text files because it’s amazing. Here is an example of a YAML file for a test environment:

name: py37_test

channels:
    - conda-forge

dependencies:
    - python=3.7
    - numpy
    - pandas
    - matplotlib

That’s it! Wasn’t that easy? The main things to remember are colons after the section names (name, channels, dependencies), indent the dashes, and add a space after dashes.

Here is an example of the dependencies section if pip is required for a specific package:

dependencies:
    - python=3.7
    - pip:
        - <pip-package-name>

I mainly use the conda-forge channel when installing packages. Some of the most useful packages only exist on the conda-forge channel. If you do list other channels – such as defaults and/or anaconda – dependency problems often pop up. It’s not the end of the world if you use multiple channels, but try to stick with one.

Create an Environment

Now it’s time to use the YAML file you just saved to create an environment. The command is straight forward:

conda env create -f <path-to-yaml-file>/py37_test_environment.yaml

Change <path-to-yaml-file> to the correct path, and make sure the YAML file name is also correct. Conda will solve the dependency list and ask if you are okay with the suggested changes. If you agree, press Enter (or Mac equivalent) . If something is wrong, press n and then Enter .

Be patient while Conda downloads and installs the packages. It might seem frozen on Executing, but if you wait long enough it should finish. Something might be wrong if you’re waiting longer than an hour for an environment to install. However, slow internet connections can add a lot of time to environment creation if large dependencies are required.

Use the Environment

Congratulations on installing a brand new environment that can be transported to any (Windows, Mac, or Linux) operating system! Now it’s time to fire it up. Open an Anaconda prompt and type the following to activate your environment:

conda activate py37_test

Super easy. Your command line should have (py37_test) at the start of the line. You’re now free to move about your environment and run python scripts!

Delete Environment

Environments can use a lot of disk space. You can simply delete the environment after a project to save space. It’s easy to re-create the environment if additional work is needed. Deleting an environment is as easy as typing the following into an Anaconda prompt:

conda remove --name py37_test --all

Conda will ask if you are sure. Press Enter to accept, or press n and then Enter to decline. Deleting an environment can take quite a while for reasons unknown to me. Don’t close the window until Conda is finished and provides a new command line.

Troubleshooting

Uh oh. Something failed. Here is a list of questions to answer if Conda failed to create the environment, from a YAML file, or if a python script failed to run within an environment:

Is the YAML file formatted correctly?
- Colon after name, channels, and dependencies
- Hyphens are indented
- Spaces after hyphens
- File extension is .yml or .yaml
Are all required packages included in the YAML file?
Does any package require a specific operating system?
Is pip required?
- If so, did you follow the pip example shown above?