My python skills aren’t super strong yet, but they’re strong enough to know that environment management is key, and it’s an area I know next to nothing about. I figured it’s time to at least learn the basics.

There seem to be two main tools for environment management in python: conda and venv. I tried to do a little bit of research to see what the pros and cons are of each. This reddit thread was useful, and this blog post linked to from that thread was helpful too. It seems like the main differences are that venv’s main purpose is for isolating a virtual environment, while conda provides more help managing packages for your project. It appears conda helps you by installing commonly used packages, which may not be suitable in industry/production settings. I’m just playing around and need all the help I can get, so I decided to take a crack at conda first.

There are two flavors of conda: anaconda and miniconda. anaconda comes with a lot more packages and also an app to manage the packages. miniconda is more stripped-down. I decided to try miniconda first, because I didn’t like the idea of installing more apps on my computer.

I’m on a Mac, and the savior for any amateur programmer on a Mac is Homebrew, a wonderful package manager for macOS which makes it so much easier to install all the little programs and utilities needed to code stuff. I can’t remember all the things I’ve installed with it, but it always seems to be there to save the day when I’m having issues installing python or R packages. Luckily it saves the day here again: you can install miniconda with Homebrew as well. This Medium article helped me figure out how. All I needed to do was run this in the Terminal:

brew install --cask miniconda
conda init zsh

Then restart the Terminal. I tested the install worked by running conda, which didn’t return any errors, which is what I wanted to see.

Now to try our first environment! I used my inky-frame project as a test case (check out my post about it here). I first wanted to try the more point-and-click way to create a python environment in VS Code. It says by using the “Python: Create Environment” command from the Control Palette, it should automatically detect an existing requirements.txt file to install the packages you need to create the environment. I tried this first, but after trying to run a script in the new environment, I got “package not found” errors, meaning it didn’t seem to actually install the necessary packages listed in requirements.txt.

Next I tried creating the conda environment using the command line by running:

conda install --file requirements.txt

But this still didn’t seem to install the packages. It gave a long error message like this:

Channels:
 - defaults
Platform: osx-64
Collecting package metadata (repodata.json): done
Solving environment: failed

LibMambaUnsatisfiableError: Encountered problems while solving:
  - nothing provides requested asttokens 2.2.1
  - nothing provides requested charset-normalizer 3.1.0
  ...

Could not solve for environment specs
The following packages are incompatible
├─ appnope 0.1.3  is installable and it requires
│  └─ python >=3.12,<3.13.0a0 , which can be installed;
├─ asttokens 2.2.1  does not exist (perhaps a typo or a missing channel);
...

(where ... means multiple lines of the same pattern for various other packages). I was stumped so I pasted this error into GitHub Copilot and asked what it meant. It told me I may need to add a “channel” for conda to search in; I only had the defaults channel added, which didn’t seem to have all the packages I was asking for. It suggested I add the conda-forge channel. I did this by running:

conda config --add channels conda-forge

After doing this, I tried to create the environment again, and I got a new but more promising error. This time it said there were only two packages it wasn’t able to install (pure-eval==0.2.2 and stack-data==0.6.2). It suggested searching https://anaconda.org to see if I could find a channel they were in that I could add. I tried searching but couldn’t find any channels they were in.

Luckily from the conda documentation I saw that you can still install packages with pip while using conda. I wasn’t sure how to tell conda to use pip using a requirements.txt file, but it seemed like it was easier using a environment.yml file instead. I asked GitHub Copilot to help me convert my requirements.txt to a environment.yml file, then tried to create the environment again with:

conda env create -f environment.yml

I still got an error, but luckily it was a small one: I needed to use single = for specifying versions for the dependencies in environment.yml, while I needed to use == for specifying versions for packages installed with pip. Fixing this, I reran the command, and finally I created a working conda environment! I also retried the point-and-click way to create an environment in VS Code, it also worked, automatically using the environment.yml file to install the necessary packages.

For the moment I’m keeping both requirements.txt and environment.yml in my repo because requirements.txt is used in the GitHub Actions I have set up. I also noticed that my GitHub Actions use different python version than what I’m using locally in this new conda environment. But I’ll take a few moments to bask in my win and will worry about that later.

Update 2024-11-10 - updating a conda environment

I’ve discovered that unless you create an immaculate environment.yml on the first try, you’ll probably have to update your conda environment. You can do that like this:

conda activate myenv #activate the environment you want to update
conda env update --file environment.yml --prune #update myenv