Can you look at the kernel.json in each of those kernelspec directories you found?

kernel.json in python2 directory:

{
 "display_name": "Python 2", 
 "language": "python", 
 "argv": [
  "python", 
  "-m", 
  "ipykernel_launcher", 
  "-f", 
  "{connection_file}"
 ]
}

kernel.json in python3 directory:

{
 "argv": [
  "python",
  "-m",
  "ipykernel_launcher",
  "-f",
  "{connection_file}"
 ],
 "display_name": "Python 3",
 "language": "python"
}

This looks similar to jupyter/jupyter#270 . Ping @minrk - I think this is due to installing kernelspecs automatically as data files in the wheel.

For now, you can probably work around it by changing the first item in argv to 'python3' in the Python 3 kernel.json file.

Yup, it's the same. Running ipython kernel install for Python 2 and 3 will resolve the ambiguity by installing with sys.executable's abspath that we can't do in data_files. While the new data_files work better in envs of all kinds and with just one Python, two Pythons running on the same prefix (e.g. --user on linux, but not mac) run into ambiguity.

I didn't fully understand what @minrk said above, but running ipython kernel install --name python3 didn't solve the issue, if that helps.

As @takluyver suggested, changing argv solved it for me, thank you. I changed the first argument of argv in python3 folder's kernel.json from python2 path to python3 path, and everything works as expected.

I got a similar problem

I solved it by sudo pip install --upgrade ipykernel

See https://stackoverflow.com/a/37857536/5658349

Solved this issue by executing

python3 -m ipykernel install --user

I also am experiencing this issue, and editing the two kernel.json files has solved the problem. Will I have to do this every time I upgrade jupyter on my machine?

Hopefully not. We've refined this behaviour in a new version of jupyter_client

This is also happening to me. Using ipython3 notebook works.

I was on vacation last week. If I recall, it was working correctly before then, and isn't working now. I don't think made any upgrades, but I could try and delete anaconda and reinstall it.

I got the same issue!
For some reason the environment is not activated in jupyter notebook. If I activate the environment in the terminal conda activate myenv and run echo $PATH, I get /Users/$USER/anaconda3/envs/myenv/bin.
However, when I start the jupyter notebook and run !echo $PATH, I get /Users/$USER/anaconda3/bin.
Therefore, also if I run which pythonin the terminal I get the python version from the environment but if I run it in the notebook I get the default python.

This is on Mac Catalina.

You might find something helpful here and in the referenced links.

Tested everything on https://ipython.readthedocs.io/en/stable/install/kernel_install.html#kernels-for-different-environments still having the same issue as @nicholas-miles

for the love of GOD, somebody please address this issue, none of the solutions work, and many have the same issue where changing the kernel does not point the python to the correct kernel environment. For me it always point into the same environment from which i ran the initial jupyter-notebook. I am stuck to that environment python, cannot change the kernel to any other one.
Within jupyter if i run this:

import sys
sys.executable

it points to the correct python environment and changes as expected, but
which python
inside the same damn jupyter notebook always point to the one directory even when i change the kernel.

It seems like the crux of the issue is that the correct python is in use by the kernel, but the PATH (from the kernel) doesn't reflect the python in use. As a result, while python statements (e.g., import foo) produce the desire results, any shell commands (e.g., !pip install foo) do not. In such cases, you could fully-qualify the shell command and, I believe, get the desire results.

I found the IPython's documentation link in the Python Environments section of the troubleshooting documentation to be fairly helpful. In particular the portion about installing the kernel with the --prefix option to point at an env different that the current env.

Jupyter, by default, doesn't activate environments for a given kernel. This is why the PATH-based commands (sans prefixing) can be problematic. If you find that activation may resolve things for you, then perhaps taking a look at nb_conda_kernels may be helpful. This package updates your jupyter's configuration to use an alternate KernelSpecManager that is used for discovering kernels (which some find invasive). These kernel specifications contain information that, AFAIK, activate the environment associated with that kernel prior to its launch.

Here's a screenshot of a kernel I've started that uses a python from an environment other than the notebook server. The notebook server is launched in my activated notebook-dev env, while the kernel I want to run is in my enterprise-gateway-dev env. Aside from the PATH and sys.executable difference, you'll find that the import statement produces the desired result (my notebook-dev env doesn't have kubernetes installed). In addition, my EG env doesn't have nose installed until I fully qualify the pip command to install it into the kernel's env (i.e., enterprise-gateway-dev).

I hope you find this helpful and apologize for the frustration.

It seems like the crux of the issue is that the correct python is in use by the kernel, but the PATH (from the kernel) doesn't reflect the python in use. As a result, while python statements (e.g., import foo) produce the desire results, any shell commands (e.g., !pip install foo) do not. In such cases, you could fully-qualify the shell command and, I believe, get the desire results.

I found this exact behavior when trying having jupyterlab to manage multiple env.

1. Each kernel have different kernel.json, each pointing different env.
2. import behave exacty as expected, only import from the pointed env
3. All !system command including !pip install and python3 --version will only point to the main kernel (in my case, default ubuntu python 3.8)

Is there any work around to install package diractly to env without leaving jupyterlab?

Hi @yasirroni - try using the %pip magic. This is exactly the problem it resolves.

ipython kernel install --user worked for me

What do I need to do to install dependency?

run pip install pandas, it installed to /usr/bin/python2 instead of /opt/python3.9

@jianboy I solved it with this

run pip install pandas, it installed to /usr/bin/python2 instead of /opt/python3.9

import sys
! {sys.executable} -m pip install pandas
import pandas

You may need to restart the kernel

The problem with which python inside notebook not pointing to the same as sys.executable is caused by a difference in system paths. In order to make sure they are pointing to the same executable:

• make sure you are using the right python version and it's pip when creating the ipykernel - ensured by python -m ...
• set up paths for your ipykernel by passing --env PATH=...

For my setup with virtualenv it looks like this

/home/user/venv/bin/python -m pip install ipython ipykernel
/home/user/venv/bin/python -m ipykernel install --user --name=kernel-name --env PATH /home/user/venv/bin:$PATH

In general:

<YOUR_PYTHON_EXECUTABLE> -m pip install ipython ipykernel
<YOUR_PYTHON_EXECUTABLE> -m ipykernel install --user --name=kernel-name --env PATH <YOUR_PYTHON_BIN_DIRECTORY>:$PATH

@nemusa Thank you so much, this worked for me! It was driving me crazy that I couldn't figure out why the PATH inside the shell didn't match my $PATH

@nemusa 's approach was the only thing that worked for me running the jupyter/minimal-notebook docker contaner