我们使用conda来维护python环境,我想了解为什么要安装google-cloud-bigquery==1.22.0,而最新可用的版本是https://pypi.org/project/google-cloud-bigquery/2.16.1/,而conda-forge (https://anaconda.org/conda-forge/google-cloud-bigquery)上的最新可验证版本是2.15.0。
下面是构建我们的conda环境的Dockerfile:
FROM debian:stretch-slim
RUN apt-get update && apt-get install curl gnupg -y && rm -rf /var/lib/apt/lists/*
SHELL ["/bin/bash", "-o", "pipefail", "-c"]
RUN curl https://repo.anaconda.com/pkgs/misc/gpgkeys/anaconda.asc | gpg --dearmor > conda.gpg && \
install -o root -g root -m 644 conda.gpg /usr/share/keyrings/conda-archive-keyring.gpg && \
gpg --keyring /usr/share/keyrings/conda-archive-keyring.gpg --no-default-keyring \
--fingerprint 34161F5BF5EB1D4BFBBB8F0A8AEB4F8B29D82806 && \
echo "deb [arch=amd64 signed-by=/usr/share/keyrings/conda-archive-keyring.gpg] https://repo.anaconda.com/pkgs/misc/debrepo/conda stable main" \
> /etc/apt/sources.list.d/conda.list
WORKDIR /tmp
RUN MINICONDA_VERSION=4.9.2 && \
CONDA_VERSION='4.9.*' && \
CONDA_DIR=/opt/conda && \
curl -O https://repo.anaconda.com/miniconda/Miniconda3-py37_${MINICONDA_VERSION}-Linux-x86_64.sh && \
/bin/bash Miniconda3-py37_${MINICONDA_VERSION}-Linux-x86_64.sh -f -b -p $CONDA_DIR && \
rm Miniconda3-py37_${MINICONDA_VERSION}-Linux-x86_64.sh && \
$CONDA_DIR/bin/conda config --system --set auto_update_conda false && \
$CONDA_DIR/bin/conda config --system --set show_channel_urls true && \
$CONDA_DIR/bin/conda config --system --remove channels defaults && \
$CONDA_DIR/bin/conda config --system --add channels main && \
$CONDA_DIR/bin/conda config --system --set env_prompt '({name}) ' && \
$CONDA_DIR/bin/conda config --system --append envs_dirs /opt/conda/envs/ && \
$CONDA_DIR/bin/conda config --system --append pkgs_dirs /opt/conda/pkgs/ && \
$CONDA_DIR/bin/conda update --quiet --yes --all conda="${CONDA_VERSION}" && \
$CONDA_DIR/bin/conda config --system --append channels conda-forge && \
$CONDA_DIR/bin/conda create -n py3 python=3.8
RUN bash -c "source /opt/conda/bin/activate /opt/conda/envs/py3 && conda install \
invoke \
apache-beam \
sh \
pytest \
pytest-xdist \
ipython \
behave \
black \
pylint \
flake8 \
jinja2 \
tenacity \
responses \
tqdm \
google-api-python-client \
google-auth-oauthlib \
google-cloud-monitoring \
google-cloud-bigquery \
google-cloud-storage \
google-cloud-pubsub \
google-cloud-secret-manager \
ipdb \
rope \
pipdeptree"我使用docker build . -t conda-env构建它,然后在容器中使用pipdeptree为google提供我的依赖树:
docker run \
--rm \
--entrypoint bash \
conda-env \
-c "source /opt/conda/bin/activate /opt/conda/envs/py3 && pipdeptree --packages google-cloud-bigquery"这给了我这个:
google-cloud-bigquery==1.22.0
- google-cloud-core [required: >=1.0.3,<2.0dev, installed: 1.6.0]
- google-api-core [required: >=1.21.0,<2.0.0dev, installed: 1.25.1]
- google-auth [required: >=1.21.1,<2.0dev, installed: 1.28.1]
- cachetools [required: >=2.0.0,<5.0, installed: 4.2.1]
- pyasn1-modules [required: >=0.2.1, installed: 0.2.8]
- pyasn1 [required: >=0.4.6,<0.5.0, installed: 0.4.8]
- rsa [required: >=3.1.4,<5, installed: 4.7.2]
- pyasn1 [required: >=0.1.3, installed: 0.4.8]
- setuptools [required: >=40.3.0, installed: 52.0.0.post20210125]
- six [required: >=1.9.0, installed: 1.15.0]
- googleapis-common-protos [required: >=1.6.0,<2.0dev, installed: 1.53.0]
- protobuf [required: >=3.12.0, installed: 3.14.0]
- six [required: >=1.9, installed: 1.15.0]
- protobuf [required: >=3.12.0, installed: 3.14.0]
- six [required: >=1.9, installed: 1.15.0]
- pytz [required: Any, installed: 2021.1]
- requests [required: >=2.18.0,<3.0.0dev, installed: 2.25.1]
- certifi [required: >=2017.4.17, installed: 2020.12.5]
- chardet [required: >=3.0.2,<5, installed: 3.0.4]
- idna [required: >=2.5,<3, installed: 2.10]
- urllib3 [required: >=1.21.1,<1.27, installed: 1.26.4]
- setuptools [required: >=40.3.0, installed: 52.0.0.post20210125]
- six [required: >=1.13.0, installed: 1.15.0]
- google-auth [required: >=1.24.0,<2.0dev, installed: 1.28.1]
- cachetools [required: >=2.0.0,<5.0, installed: 4.2.1]
- pyasn1-modules [required: >=0.2.1, installed: 0.2.8]
- pyasn1 [required: >=0.4.6,<0.5.0, installed: 0.4.8]
- rsa [required: >=3.1.4,<5, installed: 4.7.2]
- pyasn1 [required: >=0.1.3, installed: 0.4.8]
- setuptools [required: >=40.3.0, installed: 52.0.0.post20210125]
- six [required: >=1.9.0, installed: 1.15.0]
- six [required: >=1.12.0, installed: 1.15.0]
- google-resumable-media [required: >=0.3.1,<0.6.0dev,!=0.4.0, installed: 0.5.1]
- six [required: Any, installed: 1.15.0]
- protobuf [required: >=3.6.0, installed: 3.14.0]
- six [required: >=1.9, installed: 1.15.0]我必须举起我的手,承认我根本不知道该怎么解释。例如,这是否:
google-cloud-bigquery==1.22.0
- google-cloud-core [required: >=1.0.3,<2.0dev, installed: 1.6.0]这意味着google云核心正在迫使google云bigquery成为">=1.0.3,<2.0dev“,或者它是否意味着google云核心被其他东西强迫为">=1.0.3,<2.0dev”。我根本不明白这些信息是如何呈现给我的,所以我希望有人能启发我。更好的是,如果有人能告诉我如何才能安装更高版本的google-cloud-bigquery,我将非常感激,因为在1.22.0中有已知的bug。
发布于 2021-05-14 10:19:34
首先回答你的最后一个问题:
google-cloud-bigquery==1.22.0
- google-cloud-core [required: >=1.0.3,<2.0dev, installed: 1.6.0]这意味着google-cloud-bigquery是与1.22.0版本一起安装的,该版本要求在1.0.3和2.0之间安装google-cloud-core,并且安装了1.6.0版本。
要检查google-cloud-bigquery的约束--您可能会尝试这样做--添加--reverse标志,如:pipdeptree --reverse --packages google-cloud-bigquery。但是,输出并不有用,因为“您看错了树的一侧”:
Warning!!! Possibly conflicting dependencies found:
* pylint==2.7.4
- astroid [required: >=2.5.2,<2.7, installed: 2.5]
* flake8==3.9.0
- pycodestyle [required: >=2.7.0,<2.8.0, installed: 2.6.0]
- pyflakes [required: >=2.3.0,<2.4.0, installed: 2.2.0]
------------------------------------------------------------------------
google-cloud-bigquery==1.22.0因此,要查看实际的约束,运行pipdeptree --reverse并查找google-cloud-bigquery。然后您会发现,1.26.4版中的urllib3将requests约束为2.25.1,将google-api-core约束为1.25.1,将google-cloud-core约束为1.6.0,将google-cloud-bigquery约束为1.22.0。
如果我不得不猜测,我会说其中一个提到的软件包已经在它们各自的版本中安装了(编辑:甚至在安装特定的软件包之前),这将导致您所看到的结果。
我在构建的基础上运行了pip install google-cloud-bigquery --upgrade,它运行得非常好,所以您可以在最后运行它,或者在安装特定的程序之前升级所有软件包(我个人建议,更新您的基础总是一个好主意。尽管如此,如果您自己的约束“太老”,这是行不通的,您应该回到升级后,您已经安装了您的具体材料)。
https://stackoverflow.com/questions/67531666
复制相似问题