Run miniconda3 locally in Docker container

Below example has been tested on Elementary OS, but since we are using Docker, the procedure should be universal. We will be using official continuumio images

coil@coil:~$ ps -ef | grep docker
root     18991     1  0 19:09 ?        00:00:01 /usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock
coil     25760 17917  0 19:57 pts/1    00:00:00 grep --color=auto docker

coil@coil:~$ sudo docker search continuumio
[sudo] password for coil:          
NAME                                   DESCRIPTION                                     STARS               OFFICIAL            AUTOMATED
continuumio/anaconda3                  Powerful and flexible python distribution       382                                     [OK]
continuumio/anaconda                   Powerful and flexible python distribution       202                                     [OK]
continuumio/miniconda3                 Powerful and flexible package manager           138                                     [OK]
continuumio/miniconda                  Powerful and flexible package manager           67                                      [OK]
continuumio/conda_builder_linux        A build platform for Linux packages (64-bit …   4                                       
continuumio/centos5_gcc5_base                                                          3                                       
continuumio/binstar-build-linux64                                                      1                                       [OK]
atavares/miniconda-rocker-geospatial   This image is based on rocker/geospatial and…   1                                       [OK]
continuumio/anaconda2                                                                  1                                       
continuumio/conda-concourse-ci         Build and general-purpose image for Continuu…   0                                       
continuumio/anaconda-build-linux-64                                                    0                                       
continuumio/ap-auth-keycloak           Keycloak:3.1.0                                  0                                       
continuumio/miniconda2                                                                 0                                       
continuumio/ap-git-storage             gitea:1.3.0                                     0                                       
continuumio/planet-gpu                 Customized base image for Telekube that enab…   0                                       
continuumio/ci-image                                                                   0                                       
continuumio/anaconda-build-linux-32                                                    0                                       
continuumio/adam-node                                                                  0                                       
continuumio/ae-conda-packages                                                          0                                       
continuumio/sourceclear                                                                0                                       
dgenchev/continuumio                                                                   0                                       
nextjournal/python3                    Python 3 docker image for nextjournal.com. B…   0                                       [OK]
ashander/miniconda3gcc                 like continuumio/miniconda3 but with compile…   0                                       [OK]
continuumio/nginx-ingress-ws                                                           0                                       
tylercoral/bam2vcf                     Based on continuumio/miniconda3. Includes sa…   0                                       
coil@coil:~$ 
coil@coil:~$ sudo docker pull continuumio/miniconda3
[sudo] password for coil:          
Using default tag: latest
latest: Pulling from continuumio/miniconda3
1ab2bdfe9778: Already exists 
dd7d28bd8be5: Pull complete 
af998e3a361b: Pull complete 
Digest: sha256:54eb3dd4003f11f6a651b55fc2074a0ed6d9eeaa642f1c4c9a7cf8b148a30ceb
Status: Downloaded newer image for continuumio/miniconda3:latest
coil@coil:~$
coil@coil:~$ sudo docker run -i -t continuumio/miniconda3 /bin/bash
(base) root@bcb882fef67d:/# conda info

     active environment : base
    active env location : /opt/conda
            shell level : 1
       user config file : /root/.condarc
 populated config files : 
          conda version : 4.7.10
    conda-build version : not installed
         python version : 3.7.3.final.0
       virtual packages : 
       base environment : /opt/conda  (writable)
           channel URLs : https://repo.anaconda.com/pkgs/main/linux-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/linux-64
                          https://repo.anaconda.com/pkgs/r/noarch
          package cache : /opt/conda/pkgs
                          /root/.conda/pkgs
       envs directories : /opt/conda/envs
                          /root/.conda/envs
               platform : linux-64
             user-agent : conda/4.7.10 requests/2.22.0 CPython/3.7.3 Linux/4.15.0-36-generic debian/10 glibc/2.28
                UID:GID : 0:0
             netrc file : None
           offline mode : False

(base) root@bcb882fef67d:/# exit
coil@coil:~$ 

Run below command on one line. This runs docker container interactively, forwards default port 8888 on host machine to port 8888 in the container and passes commands to execute in bash that will install jupyter notebook and start it with proper parameters. Dont forget to add --allow-root option.

sudo docker run -i -t -p 8888:8888 continuumio/miniconda3 /bin/bash -c "/opt/conda/bin/conda install jupyter -y --quiet && mkdir /opt/notebooks && /opt/conda/bin/jupyter notebook --notebook-dir=/opt/notebooks --ip='*' --port=8888 --no-browser --allow-root"
coil@coil:~$ sudo docker run -i -t -p 8888:8888 continuumio/miniconda3 /bin/bash -c "/opt/conda/bin/conda install jupyter -y --quiet && mkdir /opt/notebooks && /opt/conda/bin/jupyter notebook --notebook-dir=/opt/notebooks --ip='*' --port=8888 --no-browser --allow-root"
Collecting package metadata (current_repodata.json): ...working... done
Solving environment: ...working... done

## Package Plan ##

  environment location: /opt/conda

  added / updated specs:
    - jupyter

The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    attrs-19.1.0               |           py37_1          57 KB
    backcall-0.1.0             |           py37_0          20 KB
    ...
    jinja2-2.10.1              |           py37_0         195 KB
    jpeg-9b                    |       h024ee3a_2         214 KB
    jsonschema-3.0.2           |           py37_0          92 KB
    jupyter-1.0.0              |           py37_7           6 KB
    jupyter_client-5.3.1       |             py_0          69 KB
    jupyter_console-6.0.0      |           py37_0          37 KB
    jupyter_core-4.5.0         |             py_0          48 KB
    ...
    widgetsnbextension-3.5.1   |           py37_0         862 KB
    zeromq-4.3.1               |       he6710b0_3         496 KB
    ------------------------------------------------------------
                                           Total:       133.5 MB

The following NEW packages will be INSTALLED:

  attrs              pkgs/main/linux-64::attrs-19.1.0-py37_1
  backcall           pkgs/main/linux-64::backcall-0.1.0-py37_0
  ...
  jupyter            pkgs/main/linux-64::jupyter-1.0.0-py37_7
  jupyter_client     pkgs/main/noarch::jupyter_client-5.3.1-py_0
  jupyter_console    pkgs/main/linux-64::jupyter_console-6.0.0-py37_0
  jupyter_core       pkgs/main/noarch::jupyter_core-4.5.0-py_0
  ...
  widgetsnbextension pkgs/main/linux-64::widgetsnbextension-3.5.1-py37_0
  zeromq             pkgs/main/linux-64::zeromq-4.3.1-he6710b0_3

The following packages will be UPDATED:

  ca-certificates                               2019.5.15-0 --> 2019.5.15-1
  certifi                                  2019.6.16-py37_0 --> 2019.9.11-py37_0
  conda                                       4.7.10-py37_0 --> 4.7.12-py37_0
  openssl                                 1.1.1c-h7b6447c_1 --> 1.1.1d-h7b6447c_1

Preparing transaction: ...working... done
Verifying transaction: ...working... done
Executing transaction: ...working... done
[I 18:47:52.235 NotebookApp] Writing notebook server cookie secret to /root/.local/share/jupyter/runtime/notebook_cookie_secret
[W 18:47:52.586 NotebookApp] WARNING: The notebook server is listening on all IP addresses and not using encryption. This is not recommended.
[I 18:47:52.592 NotebookApp] Serving notebooks from local directory: /opt/notebooks
[I 18:47:52.592 NotebookApp] The Jupyter Notebook is running at:
[I 18:47:52.592 NotebookApp] http://0bbdb63d7122:8888/?token=cc940d8ad6b36c3db18d62a67382245b687ea27d8d02f40a
[I 18:47:52.592 NotebookApp]  or http://127.0.0.1:8888/?token=cc940d8ad6b36c3db18d62a67382245b687ea27d8d02f40a
[I 18:47:52.592 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[C 18:47:52.600 NotebookApp] 

    To access the notebook, open this file in a browser:
        file:///root/.local/share/jupyter/runtime/nbserver-1-open.html
    Or copy and paste one of these URLs:
        http://0bbdb63d7122:8888/?token=cc940d8ad6b36c3db18d62a67382245b687ea27d8d02f40a
     or http://127.0.0.1:8888/?token=cc940d8ad6b36c3db18d62a67382245b687ea27d8d02f40a
[I 18:48:16.756 NotebookApp] 302 GET / (172.17.0.1) 1.07ms
[I 18:48:16.766 NotebookApp] 302 GET /tree? (172.17.0.1) 0.98ms
[I 18:49:29.926 NotebookApp] 302 GET /?token=cc940d8ad6b36c3db18d62a67382245b687ea27d8d02f40a (172.17.0.1) 0.73ms

We did not setup any password for login so we will need to use the access token. The token is shown at the end of notebook command line output in the container:

http://127.0.0.1:8888/?token=cc940d8ad6b36c3db18d62a67382245b687ea27d8d02f40a

Use the above link in the web browser to access the notebook (the token will be different every time jupyter notebook server is started).

Above mentioned procedure to run Jupyter notebook in the container is not ideal. Since it is installing the packages, it takes some time to start. Everytime we start the container we need to note the access token. Also all the scripts we write in the notebook are lost when container is stopped, because the container is ephemeral. So using notebook in this way is good only for quick script prototyping.

Next steps: build your own personalized container with miniconda with all the packages that you need (so they do not need to be installed every time when container is started) mount a persistent volume to the container that will keep all the scripts that you write in the jupyter notebook

Sources:

Leave a Reply

Your email address will not be published.