feat: Use RAG(Retrival-Augumented Generation) with Mistral-7b-instruct language model

Description
This pull request is to use RAG(Retrival-Augumented Generation) with a language model (Mistral-7b-instruct) in order to query the user's background information for the language model to provide a better response.

@cindyli , I replicated the error at line 53 of the rag.py script.  Bu there is another earlier error that has to do with the pydantic packages and its BaseSettings.  It looks like this is the actual error:
pydantic.errors.PydanticImportError: BaseSettings has been moved to the pydantic-settings package. See https://docs.pydantic.dev/2.7/migration/#basesettings-has-moved-to-pydantic-settings for more details.

Searching for that error led to a github issue.  There are compatibility issues with the version of python used, the version of pydantic and the version of fastapi.  A solution is referenced in a pull request to pin the versions of pydantic and fastapi to older versions.
Here's a full trace of what happened during my interactive session.  The first part uses the job_rag.sh and rag.py scripts as is, and replicates the error you reported.  The second part uses the hot-pinned solution using an earlier version of pydantic and fastapi.  But, it fails earlier at line 14 of rag.py.  It's another version issue.
First part:
[clown@cdr2598 scratch]$ module load cuda cudnn
[clown@cdr2598 scratch]$ moduel unload cuda
[mii] moduel not found! Similar commands: "mode", "modes", "modutil"
[clown@cdr2598 scratch]$ module unload cuda

Inactive Modules:
  1) cudnn     2) flexiblas     3) gcccore/.12.3     4) hwloc/2.9.1     5) libfabric/1.18.0     6) openmpi     7) pmix/4.2.4     8) ucc/1.2.0     9) ucx/1.14.1

[clown@cdr2598 scratch]$ module load StdEnv/2023

Activating Modules:
  1) flexiblas/3.3.1     2) gcccore/.12.3     3) hwloc/2.9.1     4) libfabric/1.18.0     5) openmpi/4.1.5     6) pmix/4.2.4     7) ucc/1.2.0     8) ucx/1.14.1

[clown@cdr2598 scratch]$ module load python/3.11.
[clown@cdr2598 scratch]$ module load python/3.11.5
[clown@cdr2598 scratch]$ python --version
Python 3.11.5
[clown@cdr2598 scratch]$ virtualenv --no-download $SLURM_TMPDIR/env
created virtual environment CPython3.11.5.final.0-64 in 26985ms
  creator CPython3Posix(dest=/localscratch/clown.29386918.0/env, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/home/clown/.local/share/virtualenv)
    added seed packages: pip==23.3.2, setuptools==69.2.0, wheel==0.43.0
  activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator
[clown@cdr2598 scratch]$ source $SLURM_TMPDIR/env/bin/activate
(env) [clown@cdr2598 scratch]$ pip install --upgrade pip
Looking in links: /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/gentoo2023/x86-64-v3, /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/gentoo2023/generic, /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/generic
Requirement already satisfied: pip in /localscratch/clown.29386918.0/env/lib/python3.11/site-packages (23.3.2)
Processing /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/generic/pip-24.0+computecanada-py3-none-any.whl
Installing collected packages: pip
  Attempting uninstall: pip
    Found existing installation: pip 23.3.2
    Uninstalling pip-23.3.2:
      Successfully uninstalled pip-23.3.2
Successfully installed pip-24.0+computecanada
(env) [clown@cdr2598 scratch]$ module load StdEnv/2023 rust/1.70.0 arrow/15.0.1 gcc/12.3 cudacore/.12.2.2

Activating Modules:
  1) cudnn/<IP_ADDRESS>

(env) [clown@cdr2598 scratch]$ pip install --no-index torch transformers tensorflow sentence_transformers accelerate==0.25.0 peft==0.5.0 bitsandbytes==0.42.0 datasets==2.17.0 trl
Defaulting to user installation because normal site-packages is not writeable
Looking in links: /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/gentoo2023/x86-64-v3, /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/gentoo2023/generic, /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/generic
Requirement already satisfied: torch in /home/clown/.local/lib/python3.11/site-packages (2.2.1+computecanada)
Requirement already satisfied: transformers in /home/clown/.local/lib/python3.11/site-packages (4.39.3+computecanada)
Requirement already satisfied: tensorflow in /home/clown/.local/lib/python3.11/site-packages (2.15.1+computecanada)
Requirement already satisfied: sentence_transformers in /home/clown/.local/lib/python3.11/site-packages (2.5.0+computecanada)
Requirement already satisfied: accelerate==0.25.0 in /home/clown/.local/lib/python3.11/site-packages (0.25.0+computecanada)
Requirement already satisfied: peft==0.5.0 in /home/clown/.local/lib/python3.11/site-packages (0.5.0+computecanada)
Requirement already satisfied: bitsandbytes==0.42.0 in /home/clown/.local/lib/python3.11/site-packages (0.42.0+computecanada)
Requirement already satisfied: datasets==2.17.0 in /home/clown/.local/lib/python3.11/site-packages (2.17.0+computecanada)
Requirement already satisfied: trl in /home/clown/.local/lib/python3.11/site-packages (0.7.11+computecanada)
Requirement already satisfied: numpy>=1.17 in /cvmfs/soft.computecanada.ca/easybuild/software/2023/x86-64-v3/Compiler/gcccore/scipy-stack/2023b/lib/python3.11/site-packages (from accelerate==0.25.0) (1.25.2+computecanada)
Requirement already satisfied: packaging>=20.0 in /home/clown/.local/lib/python3.11/site-packages (from accelerate==0.25.0) (23.2+computecanada)
Requirement already satisfied: psutil in /cvmfs/soft.computecanada.ca/easybuild/software/2023/x86-64-v3/Compiler/gcccore/ipykernel/2023b/lib/python3.11/site-packages (from accelerate==0.25.0) (5.9.5+computecanada)
Requirement already satisfied: pyyaml in /cvmfs/soft.computecanada.ca/easybuild/software/2023/x86-64-v3/Compiler/gcccore/scipy-stack/2023b/lib/python3.11/site-packages (from accelerate==0.25.0) (6.0.1+computecanada)
Requirement already satisfied: huggingface-hub in /home/clown/.local/lib/python3.11/site-packages (from accelerate==0.25.0) (0.22.2+computecanada)
Requirement already satisfied: safetensors>=0.3.1 in /home/clown/.local/lib/python3.11/site-packages (from accelerate==0.25.0) (0.4.1+computecanada)
Requirement already satisfied: tqdm in /home/clown/.local/lib/python3.11/site-packages (from peft==0.5.0) (4.66.2+computecanada)
Requirement already satisfied: scipy in /cvmfs/soft.computecanada.ca/easybuild/software/2023/x86-64-v3/Compiler/gcccore/scipy-stack/2023b/lib/python3.11/site-packages (from bitsandbytes==0.42.0) (1.11.2+computecanada)
Requirement already satisfied: filelock in /cvmfs/soft.computecanada.ca/easybuild/software/2023/x86-64-v3/Compiler/gcccore/python/3.11.5/lib/python3.11/site-packages (from datasets==2.17.0) (3.12.2)
Requirement already satisfied: pyarrow>=12.0.0 in /cvmfs/soft.computecanada.ca/easybuild/software/2023/x86-64-v3/CUDA/gcccore/cuda12.2/arrow/15.0.1/lib/python3.11/site-packages (from datasets==2.17.0) (15.0.1)
Requirement already satisfied: pyarrow-hotfix in /home/clown/.local/lib/python3.11/site-packages (from datasets==2.17.0) (0.6+computecanada)
Requirement already satisfied: dill<0.3.9,>=0.3.0 in /home/clown/.local/lib/python3.11/site-packages (from datasets==2.17.0) (0.3.8+computecanada)
Requirement already satisfied: pandas in /cvmfs/soft.computecanada.ca/easybuild/software/2023/x86-64-v3/Compiler/gcccore/scipy-stack/2023b/lib/python3.11/site-packages (from datasets==2.17.0) (2.1.0+computecanada)
Requirement already satisfied: requests>=2.19.0 in /cvmfs/soft.computecanada.ca/easybuild/software/2023/x86-64-v3/Compiler/gcccore/scipy-stack/2023b/lib/python3.11/site-packages (from datasets==2.17.0) (2.31.0+computecanada)
Requirement already satisfied: xxhash in /home/clown/.local/lib/python3.11/site-packages (from datasets==2.17.0) (3.2.0+computecanada)
Requirement already satisfied: multiprocess in /home/clown/.local/lib/python3.11/site-packages (from datasets==2.17.0) (0.70.16+computecanada)
Requirement already satisfied: fsspec[http]<=2023.10.0,>=2023.1.0 in /home/clown/.local/lib/python3.11/site-packages (from datasets==2.17.0) (2023.10.0+computecanada)
Requirement already satisfied: aiohttp in /home/clown/.local/lib/python3.11/site-packages (from datasets==2.17.0) (3.9.1+computecanada)
Requirement already satisfied: typing-extensions>=4.8.0 in /home/clown/.local/lib/python3.11/site-packages (from torch) (4.11.0+computecanada)
Requirement already satisfied: sympy in /cvmfs/soft.computecanada.ca/easybuild/software/2023/x86-64-v3/Compiler/gcccore/scipy-stack/2023b/lib/python3.11/site-packages (from torch) (1.12+computecanada)
Requirement already satisfied: networkx in /home/clown/.local/lib/python3.11/site-packages (from torch) (3.3+computecanada)
Requirement already satisfied: jinja2 in /cvmfs/soft.computecanada.ca/easybuild/software/2023/x86-64-v3/Compiler/gcccore/scipy-stack/2023b/lib/python3.11/site-packages (from torch) (3.1.2+computecanada)
Requirement already satisfied: regex!=2019.12.17 in /home/clown/.local/lib/python3.11/site-packages (from transformers) (2023.8.8+computecanada)
Requirement already satisfied: tokenizers<0.19,>=0.14 in /home/clown/.local/lib/python3.11/site-packages (from transformers) (0.15.0+computecanada)
Requirement already satisfied: absl-py>=1.0.0 in /home/clown/.local/lib/python3.11/site-packages (from tensorflow) (2.1.0+computecanada)
Requirement already satisfied: astunparse>=1.6.0 in /home/clown/.local/lib/python3.11/site-packages (from tensorflow) (1.6.3+computecanada)
Requirement already satisfied: flatbuffers>=23.5.26 in /home/clown/.local/lib/python3.11/site-packages (from tensorflow)<PHONE_NUMBER>5844+computecanada)
Requirement already satisfied: gast!=0.5.0,!=0.5.1,!=0.5.2,>=0.2.1 in /home/clown/.local/lib/python3.11/site-packages (from tensorflow) (0.5.4+computecanada)
Requirement already satisfied: google-pasta>=0.1.1 in /home/clown/.local/lib/python3.11/site-packages (from tensorflow) (0.2.0+computecanada)
Requirement already satisfied: h5py>=2.9.0 in /home/clown/.local/lib/python3.11/site-packages (from tensorflow) (3.10.0+computecanada)
Requirement already satisfied: libclang>=13.0.0 in /home/clown/.local/lib/python3.11/site-packages (from tensorflow) (14.0.1+computecanada)
Requirement already satisfied: ml-dtypes~=0.3.1 in /home/clown/.local/lib/python3.11/site-packages (from tensorflow) (0.3.2)
Requirement already satisfied: opt-einsum>=2.3.2 in /home/clown/.local/lib/python3.11/site-packages (from tensorflow) (3.3.0+computecanada)
Requirement already satisfied: protobuf!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<5.0.0dev,>=3.20.3 in /home/clown/.local/lib/python3.11/site-packages (from tensorflow) (4.23.4+computecanada)
Requirement already satisfied: setuptools in /cvmfs/soft.computecanada.ca/easybuild/software/2023/x86-64-v3/Compiler/gcccore/python/3.11.5/lib/python3.11/site-packages (from tensorflow) (68.1.2)
Requirement already satisfied: six>=1.12.0 in /cvmfs/soft.computecanada.ca/easybuild/software/2023/x86-64-v3/Compiler/gcccore/python/3.11.5/lib/python3.11/site-packages (from tensorflow) (1.16.0)
Requirement already satisfied: termcolor>=1.1.0 in /home/clown/.local/lib/python3.11/site-packages (from tensorflow) (2.4.0+computecanada)
Requirement already satisfied: wrapt<1.15,>=1.11.0 in /home/clown/.local/lib/python3.11/site-packages (from tensorflow) (1.14.1+computecanada)
Requirement already satisfied: tensorflow-io-gcs-filesystem>=0.23.1 in /home/clown/.local/lib/python3.11/site-packages (from tensorflow) (0.32.0+computecanada)
Requirement already satisfied: grpcio<2.0,>=1.24.3 in /home/clown/.local/lib/python3.11/site-packages (from tensorflow) (1.62.1+computecanada)
Requirement already satisfied: tensorboard<2.16,>=2.15 in /home/clown/.local/lib/python3.11/site-packages (from tensorflow) (2.15.1+computecanada)
Requirement already satisfied: tensorflow-estimator<2.16,>=2.15.0 in /home/clown/.local/lib/python3.11/site-packages (from tensorflow) (2.15.0+computecanada)
Requirement already satisfied: keras<2.16,>=2.15.0 in /home/clown/.local/lib/python3.11/site-packages (from tensorflow) (2.15.0+computecanada)
Requirement already satisfied: scikit-learn in /home/clown/.local/lib/python3.11/site-packages (from sentence_transformers) (1.3.1+computecanada)
Requirement already satisfied: Pillow in /cvmfs/soft.computecanada.ca/easybuild/software/2023/x86-64-v3/Compiler/gcccore/scipy-stack/2023b/lib/python3.11/site-packages (from sentence_transformers) (10.0.0+computecanada)
Requirement already satisfied: tyro>=0.5.11 in /home/clown/.local/lib/python3.11/site-packages (from trl) (0.7.3+computecanada)
Requirement already satisfied: wheel<1.0,>=0.23.0 in /cvmfs/soft.computecanada.ca/easybuild/software/2023/x86-64-v3/Compiler/gcccore/python/3.11.5/lib/python3.11/site-packages (from astunparse>=1.6.0->tensorflow) (0.41.2)
Requirement already satisfied: attrs>=17.3.0 in /cvmfs/soft.computecanada.ca/easybuild/software/2023/x86-64-v3/Compiler/gcccore/scipy-stack/2023b/lib/python3.11/site-packages (from aiohttp->datasets==2.17.0) (23.1.0+computecanada)
Requirement already satisfied: multidict<7.0,>=4.5 in /home/clown/.local/lib/python3.11/site-packages (from aiohttp->datasets==2.17.0) (6.0.5+computecanada)
Requirement already satisfied: yarl<2.0,>=1.0 in /home/clown/.local/lib/python3.11/site-packages (from aiohttp->datasets==2.17.0) (1.9.3+computecanada)
Requirement already satisfied: frozenlist>=1.1.1 in /home/clown/.local/lib/python3.11/site-packages (from aiohttp->datasets==2.17.0) (1.4.1+computecanada)
Requirement already satisfied: aiosignal>=1.1.2 in /home/clown/.local/lib/python3.11/site-packages (from aiohttp->datasets==2.17.0) (1.3.1+computecanada)
Requirement already satisfied: charset-normalizer<4,>=2 in /cvmfs/soft.computecanada.ca/easybuild/software/2023/x86-64-v3/Compiler/gcccore/scipy-stack/2023b/lib/python3.11/site-packages (from requests>=2.19.0->datasets==2.17.0) (3.2.0+computecanada)
Requirement already satisfied: idna<4,>=2.5 in /cvmfs/soft.computecanada.ca/easybuild/software/2023/x86-64-v3/Compiler/gcccore/scipy-stack/2023b/lib/python3.11/site-packages (from requests>=2.19.0->datasets==2.17.0) (3.4+computecanada)
Requirement already satisfied: urllib3<3,>=1.21.1 in /cvmfs/soft.computecanada.ca/easybuild/software/2023/x86-64-v3/Compiler/gcccore/scipy-stack/2023b/lib/python3.11/site-packages (from requests>=2.19.0->datasets==2.17.0) (2.0.4+computecanada)
Requirement already satisfied: certifi>=2017.4.17 in /cvmfs/soft.computecanada.ca/easybuild/software/2023/x86-64-v3/Compiler/gcccore/scipy-stack/2023b/lib/python3.11/site-packages (from requests>=2.19.0->datasets==2.17.0) (2023.7.22+computecanada)
Requirement already satisfied: google-auth<3,>=1.6.3 in /home/clown/.local/lib/python3.11/site-packages (from tensorboard<2.16,>=2.15->tensorflow) (2.25.2+computecanada)
Requirement already satisfied: google-auth-oauthlib<2,>=0.5 in /home/clown/.local/lib/python3.11/site-packages (from tensorboard<2.16,>=2.15->tensorflow) (1.2.0+computecanada)
Requirement already satisfied: markdown>=2.6.8 in /home/clown/.local/lib/python3.11/site-packages (from tensorboard<2.16,>=2.15->tensorflow) (3.5.2+computecanada)
Requirement already satisfied: tensorboard-data-server<0.8.0,>=0.7.0 in /home/clown/.local/lib/python3.11/site-packages (from tensorboard<2.16,>=2.15->tensorflow) (0.7.0+computecanada)
Requirement already satisfied: werkzeug>=1.0.1 in /home/clown/.local/lib/python3.11/site-packages (from tensorboard<2.16,>=2.15->tensorflow) (3.0.1+computecanada)
Requirement already satisfied: docstring-parser>=0.14.1 in /home/clown/.local/lib/python3.11/site-packages (from tyro>=0.5.11->trl) (0.15+computecanada)
Requirement already satisfied: rich>=11.1.0 in /home/clown/.local/lib/python3.11/site-packages (from tyro>=0.5.11->trl) (13.7.1+computecanada)
Requirement already satisfied: shtab>=1.5.6 in /home/clown/.local/lib/python3.11/site-packages (from tyro>=0.5.11->trl) (1.7.1+computecanada)
Requirement already satisfied: MarkupSafe>=2.0 in /cvmfs/soft.computecanada.ca/easybuild/software/2023/x86-64-v3/Compiler/gcccore/scipy-stack/2023b/lib/python3.11/site-packages (from jinja2->torch) (2.1.3+computecanada)
Requirement already satisfied: python-dateutil>=2.8.2 in /cvmfs/soft.computecanada.ca/easybuild/software/2023/x86-64-v3/Compiler/gcccore/ipykernel/2023b/lib/python3.11/site-packages (from pandas->datasets==2.17.0) (2.8.2+computecanada)
Requirement already satisfied: pytz>=2020.1 in /cvmfs/soft.computecanada.ca/easybuild/software/2023/x86-64-v3/Compiler/gcccore/scipy-stack/2023b/lib/python3.11/site-packages (from pandas->datasets==2.17.0) (2023.3+computecanada)
Requirement already satisfied: tzdata>=2022.1 in /cvmfs/soft.computecanada.ca/easybuild/software/2023/x86-64-v3/Compiler/gcccore/scipy-stack/2023b/lib/python3.11/site-packages (from pandas->datasets==2.17.0) (2023.3+computecanada)
Requirement already satisfied: joblib>=1.1.1 in /home/clown/.local/lib/python3.11/site-packages (from scikit-learn->sentence_transformers) (1.4.0+computecanada)
Requirement already satisfied: threadpoolctl>=2.0.0 in /home/clown/.local/lib/python3.11/site-packages (from scikit-learn->sentence_transformers) (3.4.0+computecanada)
Requirement already satisfied: mpmath>=0.19 in /cvmfs/soft.computecanada.ca/easybuild/software/2023/x86-64-v3/Compiler/gcccore/scipy-stack/2023b/lib/python3.11/site-packages (from sympy->torch) (1.3.0+computecanada)
Requirement already satisfied: cachetools<6.0,>=2.0.0 in /home/clown/.local/lib/python3.11/site-packages (from google-auth<3,>=1.6.3->tensorboard<2.16,>=2.15->tensorflow) (5.3.2+computecanada)
Requirement already satisfied: pyasn1-modules>=0.2.1 in /home/clown/.local/lib/python3.11/site-packages (from google-auth<3,>=1.6.3->tensorboard<2.16,>=2.15->tensorflow) (0.3.0+computecanada)
Requirement already satisfied: rsa<5,>=3.1.4 in /home/clown/.local/lib/python3.11/site-packages (from google-auth<3,>=1.6.3->tensorboard<2.16,>=2.15->tensorflow) (4.9+computecanada)
Requirement already satisfied: requests-oauthlib>=0.7.0 in /home/clown/.local/lib/python3.11/site-packages (from google-auth-oauthlib<2,>=0.5->tensorboard<2.16,>=2.15->tensorflow) (1.3.1+computecanada)
Requirement already satisfied: markdown-it-py>=2.2.0 in /home/clown/.local/lib/python3.11/site-packages (from rich>=11.1.0->tyro>=0.5.11->trl) (3.0.0+computecanada)
Requirement already satisfied: pygments<3.0.0,>=2.13.0 in /cvmfs/soft.computecanada.ca/easybuild/software/2023/x86-64-v3/Compiler/gcccore/ipykernel/2023b/lib/python3.11/site-packages (from rich>=11.1.0->tyro>=0.5.11->trl) (2.16.1+computecanada)
Requirement already satisfied: mdurl~=0.1 in /home/clown/.local/lib/python3.11/site-packages (from markdown-it-py>=2.2.0->rich>=11.1.0->tyro>=0.5.11->trl) (0.1.2+computecanada)
Requirement already satisfied: pyasn1<0.6.0,>=0.4.6 in /home/clown/.local/lib/python3.11/site-packages (from pyasn1-modules>=0.2.1->google-auth<3,>=1.6.3->tensorboard<2.16,>=2.15->tensorflow) (0.5.1+computecanada)
Requirement already satisfied: oauthlib>=3.0.0 in /home/clown/.local/lib/python3.11/site-packages (from requests-oauthlib>=0.7.0->google-auth-oauthlib<2,>=0.5->tensorboard<2.16,>=2.15->tensorflow) (3.2.2+computecanada)
(env) [clown@cdr2598 scratch]$ pip install langchain langchain-community unstructured chromadb
Defaulting to user installation because normal site-packages is not writeable
Looking in links: /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/gentoo2023/x86-64-v3, /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/gentoo2023/generic, /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/generic
Requirement already satisfied: langchain in /home/clown/.local/lib/python3.11/site-packages (0.1.16)
Requirement already satisfied: langchain-community in /home/clown/.local/lib/python3.11/site-packages (0.0.33)
Requirement already satisfied: unstructured in /home/clown/.local/lib/python3.11/site-packages (0.13.2)
Requirement already satisfied: chromadb in /home/clown/.local/lib/python3.11/site-packages (0.3.23)
Requirement already satisfied: PyYAML>=5.3 in /cvmfs/soft.computecanada.ca/easybuild/software/2023/x86-64-v3/Compiler/gcccore/scipy-stack/2023b/lib/python3.11/site-packages (from langchain) (6.0.1+computecanada)
Requirement already satisfied: SQLAlchemy<3,>=1.4 in /home/clown/.local/lib/python3.11/site-packages (from langchain) (2.0.29)
Requirement already satisfied: aiohttp<4.0.0,>=3.8.3 in /home/clown/.local/lib/python3.11/site-packages (from langchain) (3.9.1+computecanada)
Requirement already satisfied: dataclasses-json<0.7,>=0.5.7 in /home/clown/.local/lib/python3.11/site-packages (from langchain) (0.6.4)
Requirement already satisfied: jsonpatch<2.0,>=1.33 in /home/clown/.local/lib/python3.11/site-packages (from langchain) (1.33+computecanada)
Requirement already satisfied: langchain-core<0.2.0,>=0.1.42 in /home/clown/.local/lib/python3.11/site-packages (from langchain) (0.1.44)
Requirement already satisfied: langchain-text-splitters<0.1,>=0.0.1 in /home/clown/.local/lib/python3.11/site-packages (from langchain) (0.0.1+computecanada)
Requirement already satisfied: langsmith<0.2.0,>=0.1.17 in /home/clown/.local/lib/python3.11/site-packages (from langchain) (0.1.49)
Requirement already satisfied: numpy<2,>=1 in /cvmfs/soft.computecanada.ca/easybuild/software/2023/x86-64-v3/Compiler/gcccore/scipy-stack/2023b/lib/python3.11/site-packages (from langchain) (1.25.2+computecanada)
Requirement already satisfied: pydantic<3,>=1 in /home/clown/.local/lib/python3.11/site-packages (from langchain) (2.7.0)
Requirement already satisfied: requests<3,>=2 in /cvmfs/soft.computecanada.ca/easybuild/software/2023/x86-64-v3/Compiler/gcccore/scipy-stack/2023b/lib/python3.11/site-packages (from langchain) (2.31.0+computecanada)
Requirement already satisfied: tenacity<9.0.0,>=8.1.0 in /home/clown/.local/lib/python3.11/site-packages (from langchain) (8.2.3+computecanada)
Requirement already satisfied: chardet in /cvmfs/soft.computecanada.ca/easybuild/software/2023/x86-64-v3/Compiler/gcccore/scipy-stack/2023b/lib/python3.11/site-packages (from unstructured) (5.2.0+computecanada)
Requirement already satisfied: filetype in /home/clown/.local/lib/python3.11/site-packages (from unstructured) (1.2.0+computecanada)
Requirement already satisfied: python-magic in /home/clown/.local/lib/python3.11/site-packages (from unstructured) (0.4.27)
Requirement already satisfied: lxml in /home/clown/.local/lib/python3.11/site-packages (from unstructured) (5.1.0+computecanada)
Requirement already satisfied: nltk in /home/clown/.local/lib/python3.11/site-packages (from unstructured) (3.8.1+computecanada)
Requirement already satisfied: tabulate in /home/clown/.local/lib/python3.11/site-packages (from unstructured) (0.9.0+computecanada)
Requirement already satisfied: beautifulsoup4 in /cvmfs/soft.computecanada.ca/easybuild/software/2023/x86-64-v3/Compiler/gcccore/scipy-stack/2023b/lib/python3.11/site-packages (from unstructured) (4.12.2+computecanada)
Requirement already satisfied: emoji in /home/clown/.local/lib/python3.11/site-packages (from unstructured) (2.11.0)
Requirement already satisfied: python-iso639 in /home/clown/.local/lib/python3.11/site-packages (from unstructured) (2024.2.7)
Requirement already satisfied: langdetect in /home/clown/.local/lib/python3.11/site-packages (from unstructured) (1.0.9)
Requirement already satisfied: rapidfuzz in /home/clown/.local/lib/python3.11/site-packages (from unstructured) (3.3.0+computecanada)
Requirement already satisfied: backoff in /home/clown/.local/lib/python3.11/site-packages (from unstructured) (2.2.1+computecanada)
Requirement already satisfied: typing-extensions in /home/clown/.local/lib/python3.11/site-packages (from unstructured) (4.11.0+computecanada)
Requirement already satisfied: unstructured-client<=0.18.0 in /home/clown/.local/lib/python3.11/site-packages (from unstructured) (0.18.0)
Requirement already satisfied: wrapt in /home/clown/.local/lib/python3.11/site-packages (from unstructured) (1.14.1+computecanada)
Requirement already satisfied: pandas>=1.3 in /cvmfs/soft.computecanada.ca/easybuild/software/2023/x86-64-v3/Compiler/gcccore/scipy-stack/2023b/lib/python3.11/site-packages (from chromadb) (2.1.0+computecanada)
Requirement already satisfied: hnswlib>=0.7 in /home/clown/.local/lib/python3.11/site-packages (from chromadb) (0.8.0)
Requirement already satisfied: clickhouse-connect>=0.5.7 in /home/clown/.local/lib/python3.11/site-packages (from chromadb) (0.7.8)
Requirement already satisfied: sentence-transformers>=2.2.2 in /home/clown/.local/lib/python3.11/site-packages (from chromadb) (2.5.0+computecanada)
Requirement already satisfied: duckdb>=0.7.1 in /home/clown/.local/lib/python3.11/site-packages (from chromadb) (0.10.1+computecanada)
Requirement already satisfied: fastapi>=0.85.1 in /home/clown/.local/lib/python3.11/site-packages (from chromadb) (0.110.2)
Requirement already satisfied: uvicorn[standard]>=0.18.3 in /home/clown/.local/lib/python3.11/site-packages (from chromadb) (0.29.0)
Requirement already satisfied: posthog>=2.4.0 in /home/clown/.local/lib/python3.11/site-packages (from chromadb) (3.5.0)
Requirement already satisfied: attrs>=17.3.0 in /cvmfs/soft.computecanada.ca/easybuild/software/2023/x86-64-v3/Compiler/gcccore/scipy-stack/2023b/lib/python3.11/site-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (23.1.0+computecanada)
Requirement already satisfied: multidict<7.0,>=4.5 in /home/clown/.local/lib/python3.11/site-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (6.0.5+computecanada)
Requirement already satisfied: yarl<2.0,>=1.0 in /home/clown/.local/lib/python3.11/site-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (1.9.3+computecanada)
Requirement already satisfied: frozenlist>=1.1.1 in /home/clown/.local/lib/python3.11/site-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (1.4.1+computecanada)
Requirement already satisfied: aiosignal>=1.1.2 in /home/clown/.local/lib/python3.11/site-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (1.3.1+computecanada)
Requirement already satisfied: certifi in /cvmfs/soft.computecanada.ca/easybuild/software/2023/x86-64-v3/Compiler/gcccore/scipy-stack/2023b/lib/python3.11/site-packages (from clickhouse-connect>=0.5.7->chromadb) (2023.7.22+computecanada)
Requirement already satisfied: urllib3>=1.26 in /cvmfs/soft.computecanada.ca/easybuild/software/2023/x86-64-v3/Compiler/gcccore/scipy-stack/2023b/lib/python3.11/site-packages (from clickhouse-connect>=0.5.7->chromadb) (2.0.4+computecanada)
Requirement already satisfied: pytz in /cvmfs/soft.computecanada.ca/easybuild/software/2023/x86-64-v3/Compiler/gcccore/scipy-stack/2023b/lib/python3.11/site-packages (from clickhouse-connect>=0.5.7->chromadb) (2023.3+computecanada)
Requirement already satisfied: zstandard in /home/clown/.local/lib/python3.11/site-packages (from clickhouse-connect>=0.5.7->chromadb) (0.20.0+computecanada)
Requirement already satisfied: lz4 in /home/clown/.local/lib/python3.11/site-packages (from clickhouse-connect>=0.5.7->chromadb) (4.3.2+computecanada)
Requirement already satisfied: marshmallow<4.0.0,>=3.18.0 in /home/clown/.local/lib/python3.11/site-packages (from dataclasses-json<0.7,>=0.5.7->langchain) (3.21.1+computecanada)
Requirement already satisfied: typing-inspect<1,>=0.4.0 in /home/clown/.local/lib/python3.11/site-packages (from dataclasses-json<0.7,>=0.5.7->langchain) (0.9.0+computecanada)
Requirement already satisfied: starlette<0.38.0,>=0.37.2 in /home/clown/.local/lib/python3.11/site-packages (from fastapi>=0.85.1->chromadb) (0.37.2+computecanada)
Requirement already satisfied: jsonpointer>=1.9 in /home/clown/.local/lib/python3.11/site-packages (from jsonpatch<2.0,>=1.33->langchain) (2.4+computecanada)
Requirement already satisfied: packaging<24.0,>=23.2 in /home/clown/.local/lib/python3.11/site-packages (from langchain-core<0.2.0,>=0.1.42->langchain) (23.2+computecanada)
Requirement already satisfied: orjson<4.0.0,>=3.9.14 in /home/clown/.local/lib/python3.11/site-packages (from langsmith<0.2.0,>=0.1.17->langchain) (3.9.15+computecanada)
Requirement already satisfied: python-dateutil>=2.8.2 in /cvmfs/soft.computecanada.ca/easybuild/software/2023/x86-64-v3/Compiler/gcccore/ipykernel/2023b/lib/python3.11/site-packages (from pandas>=1.3->chromadb) (2.8.2+computecanada)
Requirement already satisfied: tzdata>=2022.1 in /cvmfs/soft.computecanada.ca/easybuild/software/2023/x86-64-v3/Compiler/gcccore/scipy-stack/2023b/lib/python3.11/site-packages (from pandas>=1.3->chromadb) (2023.3+computecanada)
Requirement already satisfied: six>=1.5 in /cvmfs/soft.computecanada.ca/easybuild/software/2023/x86-64-v3/Compiler/gcccore/python/3.11.5/lib/python3.11/site-packages (from posthog>=2.4.0->chromadb) (1.16.0)
Requirement already satisfied: monotonic>=1.5 in /home/clown/.local/lib/python3.11/site-packages (from posthog>=2.4.0->chromadb) (1.6)
Requirement already satisfied: annotated-types>=0.4.0 in /home/clown/.local/lib/python3.11/site-packages (from pydantic<3,>=1->langchain) (0.6.0+computecanada)
Requirement already satisfied: pydantic-core==2.18.1 in /home/clown/.local/lib/python3.11/site-packages (from pydantic<3,>=1->langchain) (2.18.1+computecanada)
Requirement already satisfied: charset-normalizer<4,>=2 in /cvmfs/soft.computecanada.ca/easybuild/software/2023/x86-64-v3/Compiler/gcccore/scipy-stack/2023b/lib/python3.11/site-packages (from requests<3,>=2->langchain) (3.2.0+computecanada)
Requirement already satisfied: idna<4,>=2.5 in /cvmfs/soft.computecanada.ca/easybuild/software/2023/x86-64-v3/Compiler/gcccore/scipy-stack/2023b/lib/python3.11/site-packages (from requests<3,>=2->langchain) (3.4+computecanada)
Requirement already satisfied: transformers<5.0.0,>=4.32.0 in /home/clown/.local/lib/python3.11/site-packages (from sentence-transformers>=2.2.2->chromadb) (4.39.3+computecanada)
Requirement already satisfied: tqdm in /home/clown/.local/lib/python3.11/site-packages (from sentence-transformers>=2.2.2->chromadb) (4.66.2+computecanada)
Requirement already satisfied: torch>=1.11.0 in /home/clown/.local/lib/python3.11/site-packages (from sentence-transformers>=2.2.2->chromadb) (2.2.1+computecanada)
Requirement already satisfied: scikit-learn in /home/clown/.local/lib/python3.11/site-packages (from sentence-transformers>=2.2.2->chromadb) (1.3.1+computecanada)
Requirement already satisfied: scipy in /cvmfs/soft.computecanada.ca/easybuild/software/2023/x86-64-v3/Compiler/gcccore/scipy-stack/2023b/lib/python3.11/site-packages (from sentence-transformers>=2.2.2->chromadb) (1.11.2+computecanada)
Requirement already satisfied: huggingface-hub>=0.15.1 in /home/clown/.local/lib/python3.11/site-packages (from sentence-transformers>=2.2.2->chromadb) (0.22.2+computecanada)
Requirement already satisfied: Pillow in /cvmfs/soft.computecanada.ca/easybuild/software/2023/x86-64-v3/Compiler/gcccore/scipy-stack/2023b/lib/python3.11/site-packages (from sentence-transformers>=2.2.2->chromadb) (10.0.0+computecanada)
Requirement already satisfied: greenlet!=0.4.17 in /home/clown/.local/lib/python3.11/site-packages (from SQLAlchemy<3,>=1.4->langchain) (2.0.2+computecanada)
Requirement already satisfied: dataclasses-json-speakeasy>=0.5.11 in /home/clown/.local/lib/python3.11/site-packages (from unstructured-client<=0.18.0->unstructured) (0.5.11)
Requirement already satisfied: jsonpath-python>=1.0.6 in /home/clown/.local/lib/python3.11/site-packages (from unstructured-client<=0.18.0->unstructured) (1.0.6)
Requirement already satisfied: mypy-extensions>=1.0.0 in /home/clown/.local/lib/python3.11/site-packages (from unstructured-client<=0.18.0->unstructured) (1.0.0+computecanada)
Requirement already satisfied: click>=7.0 in /home/clown/.local/lib/python3.11/site-packages (from uvicorn[standard]>=0.18.3->chromadb) (8.1.7+computecanada)
Requirement already satisfied: h11>=0.8 in /home/clown/.local/lib/python3.11/site-packages (from uvicorn[standard]>=0.18.3->chromadb) (0.14.0+computecanada)
Requirement already satisfied: httptools>=0.5.0 in /home/clown/.local/lib/python3.11/site-packages (from uvicorn[standard]>=0.18.3->chromadb) (0.5.0+computecanada)
Requirement already satisfied: python-dotenv>=0.13 in /home/clown/.local/lib/python3.11/site-packages (from uvicorn[standard]>=0.18.3->chromadb) (1.0.1+computecanada)
Requirement already satisfied: uvloop!=0.15.0,!=0.15.1,>=0.14.0 in /home/clown/.local/lib/python3.11/site-packages (from uvicorn[standard]>=0.18.3->chromadb) (0.17.0+computecanada)
Requirement already satisfied: watchfiles>=0.13 in /home/clown/.local/lib/python3.11/site-packages (from uvicorn[standard]>=0.18.3->chromadb) (0.18.0+computecanada)
Requirement already satisfied: websockets>=10.4 in /home/clown/.local/lib/python3.11/site-packages (from uvicorn[standard]>=0.18.3->chromadb) (12.0+computecanada)
Requirement already satisfied: soupsieve>1.2 in /cvmfs/soft.computecanada.ca/easybuild/software/2023/x86-64-v3/Compiler/gcccore/scipy-stack/2023b/lib/python3.11/site-packages (from beautifulsoup4->unstructured) (2.4.1+computecanada)
Requirement already satisfied: joblib in /home/clown/.local/lib/python3.11/site-packages (from nltk->unstructured) (1.4.0+computecanada)
Requirement already satisfied: regex>=2021.8.3 in /home/clown/.local/lib/python3.11/site-packages (from nltk->unstructured) (2023.8.8+computecanada)
Requirement already satisfied: filelock in /cvmfs/soft.computecanada.ca/easybuild/software/2023/x86-64-v3/Compiler/gcccore/python/3.11.5/lib/python3.11/site-packages (from huggingface-hub>=0.15.1->sentence-transformers>=2.2.2->chromadb) (3.12.2)
Requirement already satisfied: fsspec>=2023.5.0 in /home/clown/.local/lib/python3.11/site-packages (from huggingface-hub>=0.15.1->sentence-transformers>=2.2.2->chromadb) (2023.10.0+computecanada)
Requirement already satisfied: anyio<5,>=3.4.0 in /cvmfs/soft.computecanada.ca/easybuild/software/2023/x86-64-v3/Compiler/gcccore/scipy-stack/2023b/lib/python3.11/site-packages (from starlette<0.38.0,>=0.37.2->fastapi>=0.85.1->chromadb) (3.7.1+computecanada)
Requirement already satisfied: sympy in /cvmfs/soft.computecanada.ca/easybuild/software/2023/x86-64-v3/Compiler/gcccore/scipy-stack/2023b/lib/python3.11/site-packages (from torch>=1.11.0->sentence-transformers>=2.2.2->chromadb) (1.12+computecanada)
Requirement already satisfied: networkx in /home/clown/.local/lib/python3.11/site-packages (from torch>=1.11.0->sentence-transformers>=2.2.2->chromadb) (3.3+computecanada)
Requirement already satisfied: jinja2 in /cvmfs/soft.computecanada.ca/easybuild/software/2023/x86-64-v3/Compiler/gcccore/scipy-stack/2023b/lib/python3.11/site-packages (from torch>=1.11.0->sentence-transformers>=2.2.2->chromadb) (3.1.2+computecanada)
Requirement already satisfied: tokenizers<0.19,>=0.14 in /home/clown/.local/lib/python3.11/site-packages (from transformers<5.0.0,>=4.32.0->sentence-transformers>=2.2.2->chromadb) (0.15.0+computecanada)
Requirement already satisfied: safetensors>=0.4.1 in /home/clown/.local/lib/python3.11/site-packages (from transformers<5.0.0,>=4.32.0->sentence-transformers>=2.2.2->chromadb) (0.4.1+computecanada)
Requirement already satisfied: threadpoolctl>=2.0.0 in /home/clown/.local/lib/python3.11/site-packages (from scikit-learn->sentence-transformers>=2.2.2->chromadb) (3.4.0+computecanada)
Requirement already satisfied: sniffio>=1.1 in /cvmfs/soft.computecanada.ca/easybuild/software/2023/x86-64-v3/Compiler/gcccore/scipy-stack/2023b/lib/python3.11/site-packages (from anyio<5,>=3.4.0->starlette<0.38.0,>=0.37.2->fastapi>=0.85.1->chromadb) (1.3.0+computecanada)
Requirement already satisfied: MarkupSafe>=2.0 in /cvmfs/soft.computecanada.ca/easybuild/software/2023/x86-64-v3/Compiler/gcccore/scipy-stack/2023b/lib/python3.11/site-packages (from jinja2->torch>=1.11.0->sentence-transformers>=2.2.2->chromadb) (2.1.3+computecanada)
Requirement already satisfied: mpmath>=0.19 in /cvmfs/soft.computecanada.ca/easybuild/software/2023/x86-64-v3/Compiler/gcccore/scipy-stack/2023b/lib/python3.11/site-packages (from sympy->torch>=1.11.0->sentence-transformers>=2.2.2->chromadb) (1.3.0+computecanada)
(env) [clown@cdr2598 scratch]$ echo "=== Use RAG with Mistral-7b-instruct-1.0 from job $SLURM_JOB_ID on nodes $SLURM_JOB_NODELIST."
=== Use RAG with Mistral-7b-instruct-1.0 from job 29386918 on nodes cdr2598.
(env) [clown@cdr2598 scratch]$ python
Python 3.11.5 (main, Sep 18 2023, 12:23:42) [GCC 12.3.1 20230526] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch
>>> from transformers import (
...   AutoTokenizer,
...   AutoModelForCausalLM,
...   BitsAndBytesConfig,
...   pipeline
... )
2024-04-19 11:38:47.427139: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2024-04-19 11:38:51.033206: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:9261] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
2024-04-19 11:38:51.034254: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:607] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
2024-04-19 11:38:51.316939: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1515] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2024-04-19 11:38:51.826612: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
from langchain_community.document_loaders import UnstructuredFileLoader
>>> from langchain_community.document_loaders import UnstructuredFileLoader
>>> from langchain_text_splitters import RecursiveCharacterTextSplitter
>>> from langchain.embeddings.huggingface import HuggingFaceEmbeddings
>>> from langchain.prompts import PromptTemplate
>>> from langchain.schema.runnable import RunnablePassthrough
>>> from langchain.llms.huggingface_pipeline import HuggingFacePipeline
>>> from langchain.chains import LLMChain
>>> from langchain_community.vectorstores import Chroma
>>> user_files = ["/home/clown/mistral/RAG/data/user_doc.txt"]
>>> model_dir = "/home/clown/projects/ctb-whkchun/s2_bliss_LLMs/Mistral-7B-Instruct-v0.2"
>>> sentence_transformer_dir = "/home/clown/projects/ctb-whkchun/s2_bliss_LLMs/all-MiniLM-L6-v2"
>>> output_dir = "home/projects/ctb-whkchun/s2_bliss/results-rag-mistral-joseph-itest"
>>> loader = UnstructuredFileLoader(user_files, mode="elements")
>>> raw_documents = loader.load()
[nltk_data] Downloading package punkt to /home/clown/nltk_data...
[nltk_data]   Unzipping tokenizers/punkt.zip.
[nltk_data] Downloading package averaged_perceptron_tagger to
[nltk_data]     /home/clown/nltk_data...
[nltk_data]   Unzipping taggers/averaged_perceptron_tagger.zip.
>>> print(f"Loaded documents (first 10 rows):\n{raw_documents[:10]}")
Loaded documents (first 10 rows):
*** SENSITIVE, REMOVED ***
>>> embedding_func = HuggingFaceEmbeddings(model_name=sentence_transformer_dir)
>>> vectordb = Chroma.from_documents(splitted_documents, embedding_func)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'splitted_documents' is not defined
>>> splitted_documents = text_splitter.split_documents(raw_documents)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'text_splitter' is not defined
>>> splitted_documents = text_splitter.split_documents(raw_documents)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'text_splitter' is not defined
>>> text_splitter = RecursiveCharacterTextSplitter(chunk_size=200)
>>>
>>> splitted_documents = text_splitter.split_documents(raw_documents)
>>> print(f"Splitted documents (first 10 rows):\n{splitted_documents[:10]}")
Splitted documents (first 10 rows):
*** SENSITIVE, REMOVED ***
>>> vectordb = Chroma.from_documents(splitted_documents, embedding_func)
Traceback (most recent call last):
  File "/home/clown/.local/lib/python3.11/site-packages/langchain_community/vectorstores/chroma.py", line 81, in __init__
    import chromadb
  File "/home/clown/.local/lib/python3.11/site-packages/chromadb/__init__.py", line 2, in <module>
    import chromadb.config
  File "/home/clown/.local/lib/python3.11/site-packages/chromadb/config.py", line 1, in <module>
    from pydantic import BaseSettings
  File "/home/clown/.local/lib/python3.11/site-packages/pydantic/__init__.py", line 380, in __getattr__
    return _getattr_migration(attr_name)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/clown/.local/lib/python3.11/site-packages/pydantic/_migration.py", line 296, in wrapper
    raise PydanticImportError(
pydantic.errors.PydanticImportError: `BaseSettings` has been moved to the `pydantic-settings` package. See https://docs.pydantic.dev/2.7/migration/#basesettings-has-moved-to-pydantic-settings for more details.

For further information visit https://errors.pydantic.dev/2.7/u/import-error

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/clown/.local/lib/python3.11/site-packages/langchain_community/vectorstores/chroma.py", line 778, in from_documents
    return cls.from_texts(
           ^^^^^^^^^^^^^^^
  File "/home/clown/.local/lib/python3.11/site-packages/langchain_community/vectorstores/chroma.py", line 714, in from_texts
    chroma_collection = cls(
                        ^^^^
  File "/home/clown/.local/lib/python3.11/site-packages/langchain_community/vectorstores/chroma.py", line 84, in __init__
    raise ImportError(
ImportError: Could not import chromadb python package. Please install it with `pip install chromadb`.
>>> Chroma
<class 'langchain_community.vectorstores.chroma.Chroma'>
>>> import chromadb
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/clown/.local/lib/python3.11/site-packages/chromadb/__init__.py", line 2, in <module>
    import chromadb.config
  File "/home/clown/.local/lib/python3.11/site-packages/chromadb/config.py", line 1, in <module>
    from pydantic import BaseSettings
  File "/home/clown/.local/lib/python3.11/site-packages/pydantic/__init__.py", line 380, in __getattr__
    return _getattr_migration(attr_name)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/clown/.local/lib/python3.11/site-packages/pydantic/_migration.py", line 296, in wrapper
    raise PydanticImportError(
pydantic.errors.PydanticImportError: `BaseSettings` has been moved to the `pydantic-settings` package. See https://docs.pydantic.dev/2.7/migration/#basesettings-has-moved-to-pydantic-settings for more details.

For further information visit https://errors.pydantic.dev/2.7/u/import-error
>>>

Second part, using hot-pinning solution:
(env) [clown@cdr2598 scratch]$ pip show chromadb
Name: chromadb
Version: 0.3.23
Summary: Chroma.
Home-page:
Author:
Author-email: Jeff Huber<EMAIL_ADDRESS>Anton Troynikov<EMAIL_ADDRESS>License:
Location: /home/clown/.local/lib/python3.11/site-packages
Requires: clickhouse-connect, duckdb, fastapi, hnswlib, numpy, pandas, posthog, pydantic, requests, sentence-transformers, typing-extensions, uvicorn
Required-by:
(env) [clown@cdr2598 scratch]$ python --version
Python 3.11.5
(env) [clown@cdr2598 scratch]$ pip show pydantic
Name: pydantic
Version: 2.7.0
Summary: Data validation using Python type hints
Home-page:
Author:
Author-email: Samuel Colvin<EMAIL_ADDRESS>Eric Jolibois<EMAIL_ADDRESS>Hasan Ramezani<EMAIL_ADDRESS>Adrian Garcia Badaracco<EMAIL_ADDRESS>Terrence Dorsey<EMAIL_ADDRESS>David Montague<EMAIL_ADDRESS>Serge Matveenko<EMAIL_ADDRESS>Marcelo Trylesinski<EMAIL_ADDRESS>Sydney Runkle<EMAIL_ADDRESS>David Hewitt<EMAIL_ADDRESS>Alex Hall<EMAIL_ADDRESS>License:
Location: /home/clown/.local/lib/python3.11/site-packages
Requires: annotated-types, pydantic-core, typing-extensions
Required-by: chromadb, fastapi, langchain, langchain-core, langsmith
(env) [clown@cdr2598 scratch]$ pip show pydantic-settings
WARNING: Package(s) not found: pydantic-settings
(env) [clown@cdr2598 scratch]$ module spider pydantic-settings
Lmod has detected the following error:  Unable to find: "pydantic-settings".

(env) [clown@cdr2598 scratch]$ module spider pydantic

-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
  pydantic:
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
     Versions:
        pydantic/1.7.3 (E)
        pydantic/1.9.1 (E)

Names marked by a trailing (E) are extensions provided by another module.

-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
  For detailed information about a specific "pydantic" package (including how to load the modules) use the module's full name.
  Note that names that have a trailing (E) are extensions provided by other modules.
  For example:

     $ module spider pydantic/1.9.1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

(env) [clown@cdr2598 scratch]$ pip search pydantic-settings
ERROR: XMLRPC request failed [code: -32500]
RuntimeError: PyPI no longer supports 'pip search' (or XML-RPC search). Please use https://pypi.org/search (via a browser) instead. See https://warehouse.pypa.io/api-reference/xml-rpc.html#deprecated-methods for more information.
(env) [clown@cdr2598 scratch]$ pip install pydantic-settings
Defaulting to user installation because normal site-packages is not writeable
Looking in links: /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/gentoo2023/x86-64-v3, /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/gentoo2023/generic, /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/generic
Collecting pydantic-settings
  Obtaining dependency information for pydantic-settings from https://files.pythonhosted.org/packages/99/ee/24ec87e3a91426497c5a2b9880662d19cfd640342d477334ebc60fc2c276/pydantic_settings-2.2.1-py3-none-any.whl.metadata
  Downloading pydantic_settings-2.2.1-py3-none-any.whl.metadata (3.1 kB)
Requirement already satisfied: pydantic>=2.3.0 in /home/clown/.local/lib/python3.11/site-packages (from pydantic-settings) (2.7.0)
Requirement already satisfied: python-dotenv>=0.21.0 in /home/clown/.local/lib/python3.11/site-packages (from pydantic-settings) (1.0.1+computecanada)
Requirement already satisfied: annotated-types>=0.4.0 in /home/clown/.local/lib/python3.11/site-packages (from pydantic>=2.3.0->pydantic-settings) (0.6.0+computecanada)
Requirement already satisfied: pydantic-core==2.18.1 in /home/clown/.local/lib/python3.11/site-packages (from pydantic>=2.3.0->pydantic-settings) (2.18.1+computecanada)
Requirement already satisfied: typing-extensions>=4.6.1 in /home/clown/.local/lib/python3.11/site-packages (from pydantic>=2.3.0->pydantic-settings) (4.11.0+computecanada)
Downloading pydantic_settings-2.2.1-py3-none-any.whl (13 kB)
Installing collected packages: pydantic-settings
Successfully installed pydantic-settings-2.2.1
(env) [clown@cdr2598 scratch]$ python
Python 3.11.5 (main, Sep 18 2023, 12:23:42) [GCC 12.3.1 20230526] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import chromadb
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/clown/.local/lib/python3.11/site-packages/chromadb/__init__.py", line 2, in <module>
    import chromadb.config
  File "/home/clown/.local/lib/python3.11/site-packages/chromadb/config.py", line 1, in <module>
    from pydantic import BaseSettings
  File "/home/clown/.local/lib/python3.11/site-packages/pydantic/__init__.py", line 380, in __getattr__
    return _getattr_migration(attr_name)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/clown/.local/lib/python3.11/site-packages/pydantic/_migration.py", line 296, in wrapper
    raise PydanticImportError(
pydantic.errors.PydanticImportError: `BaseSettings` has been moved to the `pydantic-settings` package. See https://docs.pydantic.dev/2.7/migration/#basesettings-has-moved-to-pydantic-settings for more details.

For further information visit https://errors.pydantic.dev/2.7/u/import-error
>>> import pydantic-settings
  File "<stdin>", line 1
    import pydantic-settings
                   ^
SyntaxError: invalid syntax
>>> import pydantic_settings
>>> import chromadb
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/clown/.local/lib/python3.11/site-packages/chromadb/__init__.py", line 2, in <module>
    import chromadb.config
  File "/home/clown/.local/lib/python3.11/site-packages/chromadb/config.py", line 1, in <module>
    from pydantic import BaseSettings
  File "/home/clown/.local/lib/python3.11/site-packages/pydantic/__init__.py", line 380, in __getattr__
    return _getattr_migration(attr_name)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/clown/.local/lib/python3.11/site-packages/pydantic/_migration.py", line 296, in wrapper
    raise PydanticImportError(
pydantic.errors.PydanticImportError: `BaseSettings` has been moved to the `pydantic-settings` package. See https://docs.pydantic.dev/2.7/migration/#basesettings-has-moved-to-pydantic-settings for more details.

For further information visit https://errors.pydantic.dev/2.7/u/import-error
>>> pydantic_settings.BaseSettings
<class 'pydantic_settings.main.BaseSettings'>
>>> from pydantic_setings import BaseSettings
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'pydantic_setings'
>>> from pydantic_settings import BaseSettings
>>> import chromadb
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/clown/.local/lib/python3.11/site-packages/chromadb/__init__.py", line 2, in <module>
    import chromadb.config
  File "/home/clown/.local/lib/python3.11/site-packages/chromadb/config.py", line 1, in <module>
    from pydantic import BaseSettings
  File "/home/clown/.local/lib/python3.11/site-packages/pydantic/__init__.py", line 380, in __getattr__
    return _getattr_migration(attr_name)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/clown/.local/lib/python3.11/site-packages/pydantic/_migration.py", line 296, in wrapper
    raise PydanticImportError(
pydantic.errors.PydanticImportError: `BaseSettings` has been moved to the `pydantic-settings` package. See https://docs.pydantic.dev/2.7/migration/#basesettings-has-moved-to-pydantic-settings for more details.

For further information visit https://errors.pydantic.dev/2.7/u/import-error
>>> pip show fastapi
  File "<stdin>", line 1
    pip show fastapi
        ^^^^
SyntaxError: invalid syntax
>>>
(env) [clown@cdr2598 scratch]$ pip show fastapi
Name: fastapi
Version: 0.110.2
Summary: FastAPI framework, high performance, easy to learn, fast to code, ready for production
Home-page:
Author:
Author-email: Sebastián Ramírez<EMAIL_ADDRESS>License:
Location: /home/clown/.local/lib/python3.11/site-packages
Requires: pydantic, starlette, typing-extensions
Required-by: chromadb
(env) [clown@cdr2598 scratch]$ pip uninstall pydantic-settings
Found existing installation: pydantic-settings 2.2.1
Uninstalling pydantic-settings-2.2.1:
  Would remove:
    /home/clown/.local/lib/python3.11/site-packages/pydantic_settings-2.2.1.dist-info/*
    /home/clown/.local/lib/python3.11/site-packages/pydantic_settings/*
Proceed (Y/n)? y
  Successfully uninstalled pydantic-settings-2.2.1
(env) [clown@cdr2598 scratch]$ pip uninstall pydantic
Found existing installation: pydantic 2.7.0
Uninstalling pydantic-2.7.0:
  Would remove:
    /home/clown/.local/lib/python3.11/site-packages/pydantic-2.7.0.dist-info/*
    /home/clown/.local/lib/python3.11/site-packages/pydantic/*
Proceed (Y/n)? y
  Successfully uninstalled pydantic-2.7.0
(env) [clown@cdr2598 scratch]$ pip uninstall fastapi
Found existing installation: fastapi 0.110.2
Uninstalling fastapi-0.110.2:
  Would remove:
    /home/clown/.local/lib/python3.11/site-packages/fastapi-0.110.2.dist-info/*
    /home/clown/.local/lib/python3.11/site-packages/fastapi/*
Proceed (Y/n)? y
  Successfully uninstalled fastapi-0.110.2
(env) [clown@cdr2598 scratch]$ pip install pydantic==1.9
Defaulting to user installation because normal site-packages is not writeable
Looking in links: /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/gentoo2023/x86-64-v3, /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/gentoo2023/generic, /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/generic
Collecting pydantic==1.9
  Obtaining dependency information for pydantic==1.9 from https://files.pythonhosted.org/packages/d4/4e/00724eebf52854e65dabe2c190b4842afbda0e09817f415683a3130a123c/pydantic-1.9.0-py3-none-any.whl.metadata
  Using cached pydantic-1.9.0-py3-none-any.whl.metadata (121 kB)
Requirement already satisfied: typing-extensions>=<IP_ADDRESS> in /home/clown/.local/lib/python3.11/site-packages (from pydantic==1.9) (4.11.0+computecanada)
Downloading pydantic-1.9.0-py3-none-any.whl (140 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 140.3/140.3 kB 317.1 kB/s eta 0:00:00
Installing collected packages: pydantic
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
chromadb 0.3.23 requires fastapi>=0.85.1, which is not installed.
Successfully installed pydantic-1.9.0
(env) [clown@cdr2598 scratch]$ pip install fastapi==0.85.1
Defaulting to user installation because normal site-packages is not writeable
Looking in links: /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/gentoo2023/x86-64-v3, /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/gentoo2023/generic, /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/generic
Collecting fastapi==0.85.1
  Obtaining dependency information for fastapi==0.85.1 from https://files.pythonhosted.org/packages/bf/54/6eb1882b5cb29e6647df92ee74d0a93dab149234ec914563cab955fa667f/fastapi-0.85.1-py3-none-any.whl.metadata
  Using cached fastapi-0.85.1-py3-none-any.whl.metadata (24 kB)
Requirement already satisfied: pydantic!=1.7,!=1.7.1,!=1.7.2,!=1.7.3,!=1.8,!=1.8.1,<2.0.0,>=1.6.2 in /home/clown/.local/lib/python3.11/site-packages (from fastapi==0.85.1) (1.9.0)
Collecting starlette==0.20.4 (from fastapi==0.85.1)
  Obtaining dependency information for starlette==0.20.4 from https://files.pythonhosted.org/packages/51/37/8ac52116984d6a0d8502ec2c7e4a4a78f862b76410cdb1a4bcb384c91cb3/starlette-0.20.4-py3-none-any.whl.metadata
  Downloading starlette-0.20.4-py3-none-any.whl.metadata (5.5 kB)
Requirement already satisfied: anyio<5,>=3.4.0 in /cvmfs/soft.computecanada.ca/easybuild/software/2023/x86-64-v3/Compiler/gcccore/scipy-stack/2023b/lib/python3.11/site-packages (from starlette==0.20.4->fastapi==0.85.1) (3.7.1+computecanada)
Requirement already satisfied: typing-extensions>=<IP_ADDRESS> in /home/clown/.local/lib/python3.11/site-packages (from pydantic!=1.7,!=1.7.1,!=1.7.2,!=1.7.3,!=1.8,!=1.8.1,<2.0.0,>=1.6.2->fastapi==0.85.1) (4.11.0+computecanada)
Requirement already satisfied: idna>=2.8 in /cvmfs/soft.computecanada.ca/easybuild/software/2023/x86-64-v3/Compiler/gcccore/scipy-stack/2023b/lib/python3.11/site-packages (from anyio<5,>=3.4.0->starlette==0.20.4->fastapi==0.85.1) (3.4+computecanada)
Requirement already satisfied: sniffio>=1.1 in /cvmfs/soft.computecanada.ca/easybuild/software/2023/x86-64-v3/Compiler/gcccore/scipy-stack/2023b/lib/python3.11/site-packages (from anyio<5,>=3.4.0->starlette==0.20.4->fastapi==0.85.1) (1.3.0+computecanada)
Downloading fastapi-0.85.1-py3-none-any.whl (55 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 55.4/55.4 kB 9.7 kB/s eta 0:00:00
Downloading starlette-0.20.4-py3-none-any.whl (63 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 63.6/63.6 kB 11.4 kB/s eta 0:00:00
Installing collected packages: starlette, fastapi
  Attempting uninstall: starlette
    Found existing installation: starlette 0.37.2+computecanada
    Uninstalling starlette-0.37.2+computecanada:
      Successfully uninstalled starlette-0.37.2+computecanada
Successfully installed fastapi-0.85.1 starlette-0.20.4
(env) [clown@cdr2598 scratch]$ python
Python 3.11.5 (main, Sep 18 2023, 12:23:42) [GCC 12.3.1 20230526] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import chromadb
>>>
(env) [clown@cdr2598 scratch]$ python ~/mistral/RAG/rag.py
2024-04-19 13:22:23.079869: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2024-04-19 13:22:26.332817: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:9261] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
2024-04-19 13:22:26.333672: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:607] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
2024-04-19 13:22:26.543526: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1515] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2024-04-19 13:22:26.936112: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
Traceback (most recent call last):
  File "/home/clown/mistral/RAG/rag.py", line 14, in <module>
    from langchain.prompts import PromptTemplate
  File "/home/clown/.local/lib/python3.11/site-packages/langchain/prompts/__init__.py", line 30, in <module>
    from langchain_core.example_selectors import (
  File "/home/clown/.local/lib/python3.11/site-packages/langchain_core/example_selectors/__init__.py", line 6, in <module>
    from langchain_core.example_selectors.length_based import (
  File "/home/clown/.local/lib/python3.11/site-packages/langchain_core/example_selectors/length_based.py", line 6, in <module>
    from langchain_core.prompts.prompt import PromptTemplate
  File "/home/clown/.local/lib/python3.11/site-packages/langchain_core/prompts/__init__.py", line 27, in <module>
    from langchain_core.prompts.base import (
  File "/home/clown/.local/lib/python3.11/site-packages/langchain_core/prompts/base.py", line 22, in <module>
    from langchain_core.output_parsers.base import BaseOutputParser
  File "/home/clown/.local/lib/python3.11/site-packages/langchain_core/output_parsers/__init__.py", line 33, in <module>
    from langchain_core.output_parsers.xml import XMLOutputParser
  File "/home/clown/.local/lib/python3.11/site-packages/langchain_core/output_parsers/xml.py", line 127, in <module>
    class XMLOutputParser(BaseTransformOutputParser):
  File "/home/clown/.local/lib/python3.11/site-packages/pydantic/main.py", line 204, in __new__
    fields[ann_name] = ModelField.infer(
                       ^^^^^^^^^^^^^^^^^
  File "/home/clown/.local/lib/python3.11/site-packages/pydantic/fields.py", line 488, in infer
    return cls(
           ^^^^
  File "/home/clown/.local/lib/python3.11/site-packages/pydantic/fields.py", line 419, in __init__
    self.prepare()
  File "/home/clown/.local/lib/python3.11/site-packages/pydantic/fields.py", line 539, in prepare
    self.populate_validators()
  File "/home/clown/.local/lib/python3.11/site-packages/pydantic/fields.py", line 801, in populate_validators
    *(get_validators() if get_validators else list(find_validators(self.type_, self.model_config))),
                                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/clown/.local/lib/python3.11/site-packages/pydantic/validators.py", line 723, in find_validators
    raise RuntimeError(f'no validator found for {type_}, see `arbitrary_types_allowed` in Config')
RuntimeError: no validator found for <class 're.Pattern'>, see `arbitrary_types_allowed` in Config


Example doesn't work.

At the stage 'Run the function locally' I get the error:  Synchronous operations are disallowed. Call WriteAsync or set AllowSynchronousIO to true instead.
Using VS 17.9.5

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 6c557865-c800-2846-1d10-003f2c16213b
Version Independent ID: a30de769-074f-bb42-cd76-3397598069b2
Content: Connect functions to Azure Storage using Visual Studio
Content Source: articles/azure-functions/functions-add-output-binding-storage-queue-vs.md
Service: azure-functions
GitHub Login: @ggailey777
Microsoft Alias: glenga

@JasperKent
Thanks for your feedback! We will investigate and update as appropriate.


Question: LVQ feature dropped?

This is not an issue but more a question. I see LVQ feature is removed completely in the commit https://github.com/jbellis/jvector/commit/9126f0eff18c8bcfff75cc1384e9a23258ec4e08, but there's no description or explanation in the commit about that.
What was the issue? Isn't it useful or not working nice in JVector?

I am also in the dark. @jkni ?


Perf2

Summary

If merged this pull request will improve performance and is discussed in #155
Proposed changes

See #155

Please note I did disable pylints R0902, let me know if we want to change something as well. It would also appear I broke some of the unit tests, so I will need to fix those as well.

@nightlark I believe this one is ready for review.

Actually, one question -- does https://github.com/LLNL/Surfactant/blob/main/surfactant/cmd/merge.py for the merge subcommand need some changes to work with the sets (append -> add) and maintain the sha256 lookup table?

Yes, I think there is only one change needed on line 61, but I will make sure and make edits tomorrow.

I was thinking line 61, and two things in https://github.com/LLNL/Surfactant/blob/e2036bb62febeaa5c71913b2acac5fd5f2b6070f/surfactant/sbomtypes/_sbom.py#L120

a call to relationships.append() in the merge function
not updating the internal sha256 hashmap with software entries added via the merge function (if the entry to be added has a sha256 that isn’t None)
the add_software function above the merge function in the SBOM class maybe should also update the hashmap (if the software entry has a sha256 hash)

I wonder if maybe the reason it didn’t break is that the merge function mostly uses the “internal” _ find functions that aren’t using the hashmap?

We might want to consider updating the input reader for the cytrics format to rebuild the sha256 hashmap when loading a new SBOM. I don't think this affects any of the functionality built-into Surfactant though, since reading existing SBOMs isn't used much (and where it is, the sha256 lookup table is not used).


Landing Page Design Rework

so i have been making a landing page design

Can you look into and see if you are ok with it.
if everything is ok i can work on it

First of all thanks for taking the time to improve our first time visitor experience. Here are my thoughts about all segments. @Clashsoft I would like you to expand on this and give your own 2cents. I try to give my feedback from the perspective of a first time user. I'll make numbered lists for easier reference here in the discussion.
General

I think instead of Real-Time usage we should advertise the 100% free to use feature. e.g. like it's done in the About page.
Was darkmode considered?

Navbar

A lot of the information that we currently have in our navbar has been lost. There are no recent polls, changelog and so on. I would suggest that we keep the current navbar, as we have way more information that could be visually displayed in a simple navbar without any burger menus.
The thing that this navbar does better is highlighting the option to sign in for the first time users. We need to re-consider if we highlight this button when the user isn't signed in. I'll leave this up for discussion.

First info Segment
(Headline, Subtext and Create a Poll-Button)
It shows clear intent and summarizes the basic fuctionalities in a headline. Furthermore there is some additional information that highlights the efficiency of this tool. I think it looks clear for users.
Example Screenshot with Text
Always a good idea to showcase some usage from the application!

The text right next to it needs to be worked upon and I'm not sure if we should use muted /greyed out text there, since it looks a bit off in my opinion.

Cards

Cards look a bit uninteresting, because they are completly greyed out. Is there some idea behind it that we will do something with them on:hover?
Maybe add some icons?

Coperate Icons
(I think that's how they are called, but I'm not sure)

The overall style looks good, but it just repeats the information we were given several times before.
Do we have a license for these icons? Are they generated?

Footer

Generally a good idea to have one, but we don't have any social accounts that we could link to. It could make sense to add legal info etc. there, but on the other side that info is already given in the navbar.

https://www.figma.com/proto/RcB4OmiSruBhKwidKA9ZYm/Apollusia?node-id=15-3&node-type=frame&t=dZ2dkNgMpUw1xcH3-1&scaling=min-zoom&content-scaling=fixed&page-id=0%3A1
So made some changes, and moved the features in the sidebar to the footer. if this is ok ill start dev

So made some changes, and moved the features in the sidebar to the footer

The burgermenu / navbar shouldn't be changed for now. I think we can improve this in a subsequent PR / Issue if necessary. Moving the items to the footer has also lost the information for all "Recent Polls". I think it would also make more sense to refactor it later, if needed at all, since I currently see no issues with the current navbar except the possible improvement of the Sign in button being made more clear. But we will need to validate that with some users first.

if this is ok I'll start dev.

It would be great if you could wait for @Clashsoft reply, since he is the other maintainer that needs to approve the changes. Afterwards we need to clarify all points first, before we start the development to avoid unwanted design changes.

if you want I can use some other icons , that you suggest

We currently use bootstrap-icons in our project (see package.json).

ok so what if i just kept the sidebar too? would that make it ok since i wont miss anything else either

Yes, you should keep the sidebar. I don't mind adding stuff to the navbar either as long as it works in mobile.
Regarding the design:

The copy (text blocks) is sometimes a little wonky or even wrong. Ex: "Customize your poll with different question types", "Choose from various question formats like multiple choice, ratings to match your needs". Apollusia is not a generic poll platform, it's just for finding time slots. Maybe we should just highlight a few of the features in the Readme/About.
"How it works" is almost unreadable
Other than that it seems fine. If you start working on it, I suggest you create a new page like home and place it under the / route, so you don't have to change the dashboard (which you call My Polls)

Ok I'll do that


add: system76-power-deb

It also doesn't works in debian, I guess that may be the reason for not being in the repo.


fix: security vulnerability

:tada: This PR is included in version 1.16.3 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


add tests for algob sign-multisig

Proposed Changes
Added tests for algob sign-multisig command. Removed a TODO from previous test.

do we have test for signMultiSig function ?
this doesn't seems to test that.

with new types i cannot read msig property here.
could you please try it with new types ?

I think we discussed that decodeSignedTransaction should return msig property. In tests with latest algosdk (although without types) i am getting the txn.msig property (as expected). So it seems like a types issue right (same as https://github.com/algorand/js-algorand-sdk/issues/389)?

I think we discussed that decodeSignedTransaction should return msig property. In tests with latest algosdk (although without types) i am getting the txn.msig property (as expected). So it seems like a types issue right (same as algorand/js-algorand-sdk#389)? Msig should be there in new types

I know we discussed, but need to confirm does this work here.


Add trace in import log when the overviews process starts

Otherwise, this might take some time and it would not be noticeable from the logs until it finishes.

Actually, there exists a trace here: https://github.com/CartoDB/cartodb/blob/3e178a5ce26f2d7fd7772537d2fe9013eb56e2ce/services/importer/lib/importer/overviews.rb#L50 but it is not stored.

https://github.com/CartoDB/cartodb/blob/289e901b556852eb1682627c34c633c99c06bca6/app/connectors/importer.rb#L52


[CLOSED] clustering

 Issue by jeff-regier
Wednesday Apr 04, 2018 at 03:57 GMT
Originally opened as https://github.com/YosefLab/scVI-dev/issues/14

To start, maybe figure out when VaDE does/doesn't work.
https://arxiv.org/pdf/1611.05148.pdf

 Comment by jeff-regier
Wednesday Apr 18, 2018 at 18:22 GMT

@maxime1310 How about storing your work on this on a branch, e.g. max/vade, and then closing the issue?

 Comment by maxime1310
Wednesday Apr 18, 2018 at 19:38 GMT

@jeff-regier I'll just clean the code and make sure it's able to find the right clusters with the Retina dataset (for now it runs fine on Cortex), and then I'll do this so we can close the issue!

 Comment by jeff-regier
Friday Apr 20, 2018 at 00:25 GMT

Sounds good, thanks Maxime.

 Comment by maxime1310
Friday Apr 20, 2018 at 00:03 GMT

@jeff-regier the code I just pushed should:
-reproduce Romain's visual clustering on the Retina dataset
-show if using a VADE with those pre-trained weights yields improvement in the clustering (haven't included clustering metrics yet, the appreciation is simply visual for the moment)
I launched it for the same number of epochs as Romain did (it takes a bit of time as the dataset is huge), once I have satisfying visual clustering results I'll push it and close the issue.

 Comment by jeff-regier
Tuesday Apr 24, 2018 at 18:03 GMT

Very interesting. Even though there's no obvious improvement for VADE, it could just be that this dataset is too easy to cluster (and that the few mistakes we see are essentially impossible to correct). Might be good to revisit VADE if we need to improve visualization in the future.

 Comment by maxime1310
Tuesday Apr 24, 2018 at 17:50 GMT

The VADE doesn't improves much the clustering metrics used in the paper (i.e ARI, NMI, silhouette score) on the Retina dataset, but on the other hand (interesting fact considering the desperate need for pretraining on MNIST) seems to also work without much pretraining.
After pretraining:

After VADE:

 Comment by maxime1310
Tuesday Apr 24, 2018 at 18:08 GMT

I definitely agree with you. Perhaps I'll take a bit of time to run it on datasets where scVI performs less well for clustering to see if there is improvement.


Token search does not implement filtering by system

Reference in the spec: https://hl7.org/fhir/search.html#token
We should be able to filter tokens using the system; right now, only the code seem to work.
Our use case is to display conditions in different sections based on the system of the coding.
Example - searching for ICD-10 coded Conditions:
/Condition?code=http://hl7.org/fhir/sid/icd-10-cm|
At the moment, this returns nothing as it probably look for codeof value http://hl7.org/fhir/sid/icd-10-cm|.

cc @rahul1


Suggestion for X button - Hide instead of close

Env - Ubuntu 16.04
Just like other chat applications (skype for example), the X button could just hide the application instead of closing it. The user can close the application from the try
Just a suggestion of usability, feel free to close it if not interested.
Cheers
Luca

Dunno if on Linux version too. But on windows version, when you go to settings you can turn on this behaviour of hiding instread of turning off.


error":{"code":-32003

Transaction sim:
{"jsonrpc":"2.0","error":{"code":-32003,"message":"Transaction signature verification failure"},"id":1}
I cant mint nft.
This is my code:
`
       var rpcClient = ClientFactory.GetClient(Cluster.MainNet);
        var blockHash = rpcClient.GetRecentBlockHash();
        var wallet = new Wallet(MnemonicWords);

        var fromAccount = wallet.Account;
        var mintAccount = wallet.GetAccount(0);
        var tokenAccount = wallet.GetAccount(0);

        byte[] metadataAddress = new byte[32];
        int nonce;
        AddressExtensions.TryFindProgramAddress(
            new List<byte[]>() {
                Encoding.UTF8.GetBytes("metadata"),
                MetadataProgram.ProgramIdKey,
                mintAccount.PublicKey
            },
            MetadataProgram.ProgramIdKey,
            out metadataAddress,
            out nonce
        );

        Console.WriteLine($"PDA METADATA: {new PublicKey(metadataAddress)}");
        byte[] masterEditionAddress = new byte[32];
        //int nonce;
        AddressExtensions.TryFindProgramAddress(
            new List<byte[]>() {
                Encoding.UTF8.GetBytes("metadata"),
                MetadataProgram.ProgramIdKey,
                mintAccount.PublicKey
            },
            MetadataProgram.ProgramIdKey,
            out masterEditionAddress,
            out nonce
        );
        Console.WriteLine($"PDA MASTER: {new PublicKey(masterEditionAddress)}");

        var c1 = new Creator(fromAccount.PublicKey, 100);

        var data = new MetadataParameters()
        {
            name = "Test 2",
            symbol = "",
            image = "https://i.resmim.net/hddxqQ.jpg",
            uri = "http://foodi.com",
            creators = new List<Creator>() { c1 },
            sellerFeeBasisPoints = 5
        };

        byte[] TX2 = new TransactionBuilder()
            .SetRecentBlockHash(blockHash.Result.Value.Blockhash)
            .SetFeePayer(fromAccount)
            .AddInstruction(
                MetadataProgram.CreateMetadataAccount(
                    new PublicKey(metadataAddress), //PDA
                    mintAccount.PublicKey,  //MINT
                    fromAccount.PublicKey,  //mint AUTHORITY
                    fromAccount.PublicKey,  //PAYER
                    fromAccount.PublicKey,
                    data,                   //DATA
                    true,
                    false                    //ISMUTABLE
                )
            )
            .AddInstruction(
                MetadataProgram.PuffMetada(
                    new PublicKey(metadataAddress)
                )
            )
            .AddInstruction(
                MetadataProgram.CreateMasterEdition(
                    1,
                    new PublicKey(masterEditionAddress),
                    mintAccount.PublicKey,
                    fromAccount.PublicKey,
                    fromAccount.PublicKey,
                    fromAccount.PublicKey,
                    new PublicKey(metadataAddress)
                )
            )
            .Build(new List<Account> { fromAccount, wallet.GetAccount(0) });
        var txSim2 = rpcClient.SendTransaction(TX2);
        Console.WriteLine($"Transaction sim: \n { txSim2.RawRpcResponse }");`

var fromAccount = wallet.Account;
var mintAccount = wallet.GetAccount(0);
var tokenAccount = wallet.GetAccount(0);

            .Build(new List<Account> { fromAccount, wallet.GetAccount(0) });

You are signing the transaction with the same key twice.

var fromAccount = wallet.Account;
var mintAccount = wallet.GetAccount(0);
var tokenAccount = wallet.GetAccount(0);

            .Build(new List<Account> { fromAccount, wallet.GetAccount(0) });

You are signing the transaction with the same key twice.

i removed one of them. And now im getting another error. I think there is progress..
If you can help me, i will be appreciate.
 {"jsonrpc":"2.0","error":{"code":-32002,"message":"Transaction simulation failed: Error processing Instruction 0: Program failed to complete","data":{"accounts":null,"err":{"InstructionError":[0,"ProgramFailedToComplete"]},"logs":["Program metaqbxxUerdq28cj1RbAWkYQm3ybzjb6a8bt518x1s invoke [1]","Program log: Instruction: Create Metadata Accounts","Program log: libstd rust_begin_panic","Program log: panicked at 'range end index 36 out of range for slice of length 0', token-metadata/program/src/utils.rs:234:27","Program metaqbxxUerdq28cj1RbAWkYQm3ybzjb6a8bt518x1s consumed 200000 of 200000 compute units","Program failed to complete: BPF program panicked","Program metaqbxxUerdq28cj1RbAWkYQm3ybzjb6a8bt518x1s failed: Program failed to complete"]}},"id":1}

If you look a the MetadaTest.cs code you will see that the mint and token account are created in a prior transaction. Seeing you are using the same account as a fromAccount and mintAccount, I have to ask did you create a mint first?

No, i actually dont know how nft system is work. im really trying to understand. do you have example code? it can help me to understand.

If you look a the MetadataTest.cs code you will see that the mint and token account are created in a prior transaction. Seeing you are using the same account as a fromAccount and mintAccount, I have to ask did you create a mint first?

NFTs on Solana work as SPL tokens with 0 decimals and distribution of only 1 token. After that , you create a metadata account that holds the metadata and then master edition which controls the "printing of NFTs" ( maybe you want more then edition as a poster print ie). So its : create a mint-> mint 1 token -> create metadata account -> create masterEdition account. All that gives you a NFT.
In the Solnet.Metaplex.Test/Metadata.cs you will find two test functions: MintToken and TestCreateMetadataAccount. They are both needed to make a NFT. It simply doesn't fit in 200000 compute operations constraint of a transaction.
You can find out more at http://docs.metaplex.com and Metaplexes discord.

I understand everything and now i can create nft with no problem. First, thank you both @murlokito and @mariomatic.
Now i have a new question. I really understand everything but i dont understand how i create nft with image?
Im trying send bytes with my image and it doesnt work.
`
internal static byte[] EncodeCreateMetadataAccountData( MetadataParameters parameters,  bool isMutable = true  )
{
byte[] encodedName = Serialization.EncodeRustString(parameters.name);
byte[] encodedSymbol = Serialization.EncodeRustString(parameters.symbol);
byte[] encodedUri = Serialization.EncodeRustString(parameters.uri);
        var buffer = new MemoryStream();
        BinaryWriter writer = new BinaryWriter(buffer);

        //  I download my image to byte array
        var webClient = new WebClient();
        byte[] imageBytes = webClient.DownloadData(parameters.image);

         //  encodedImage is Encode Rust image
        byte[] encodedImage = new byte[imageBytes.Length + sizeof(uint)];
        encodedImage.WriteU32((uint)imageBytes.Length, 0);
        encodedImage.WriteSpan(imageBytes, sizeof(uint));

        writer.Write((byte)MetadataProgramInstructions.Values.CreateMetadataAccount);
        writer.Write(encodedName);
        writer.Write(encodedSymbol);

        writer.Write(encodedUri);
        writer.Write((ushort)parameters.sellerFeeBasisPoints);

        if (parameters.creators == null || parameters.creators?.Count < 1)
        {
            writer.Write(new byte[] { 0 }); //Option()
        }
        else
        {
            writer.Write((byte)1);
            writer.Write(parameters.creators.Count);
            foreach (Creator c in parameters.creators)
            {
                byte[] encodedCreator = c.Encode();
                writer.Write(encodedCreator);
            }
        }

       //here i write my encoded image.
        writer.Write(encodedImage);
        writer.Write(isMutable);
        return buffer.ToArray();
    }`

You upload your file somewhere ( Metaplex uploads it to Arweave ). Then you  put that uri link inside of a json, and then upload a json somewhere, and then put the link to the json inside of data uri field. Yes , it's complicated...
So, no we don't put the media data inside of the metadata ( that would be an atomic nft, I think only Arweave has them ).
You can read about the nft json standard:
https://docs.metaplex.com/nft-standard

Solnet library has a code for upload json? How can i integrate json with solnet.metaplex? if you have an example code for it, i will be appreciated


FYI: user_pref("dom.webnotifications.enabled", false); can cause site breakage

user_pref("dom.webnotifications.enabled", false);
prevented an online shop to display product images.
It took me a while to troubleshoot this issue so I decided to share. I thought this pref has nothing to do with image content loading. The URL is not popular internationally, so I won't share it. Pants you can have it via email if you're interested.

Sure - email it to me - is it sensitive for some reason .. have you been looking at buttplugs and gimp suits? Or are you afraid it might give away your country? Honestly, I'm okay with the sexual fetishes, I don't even mind if it's a MLP specialty site, or even Donald Trump's merchandise for overpriced bottled water and cheap-ass useless university degrees. I am not here to judge, bro .. just sayin' :grin:
Any reason I can't post it here if I need help from others? Let me know in the email

Hilarious! Please check your inbox.

OK, got it. Soz for the delay - I am a bit swamped for the next few days. Not even what I can do, but will get around to it

Two issues:

you said the site requires dom storage and webnotifications to function properly (display images)
you asked if it's OK to have dom webnotifications enabled permanently

2
I am not 100% sure on all this workers stuff. I thought (and the documentation said so) that there was some sort of chain here: "Push and web notifications require service workers, which in turn require workers." (from the user.js). But I have seen youtube behave differently when just service workers was allowed - see #234 (so clearly dom.workers does NOT control service workers). Maybe the same applies to notifications.
See 2304 in the user.js - you can allow site exceptions - so there is no need to default allow. As for whether of not this works without allowing serviceworkers, IDK. There are 2 prefs listed and the dom.webnotifications.serviceworker.enabled implies by its name that it has a dependency on serviceworkers being enabled. As for dom.webnotifications.enabled, IDK. I also have no real idea on what the site exception option "receive notifications" refers to - as my default seems to be "always ask" - I think this UI may be broken a bit like Maintain Offsite Storage (IDB) is, which BTW is removed in 58.
Add to this the fact that dom.push.* notifications (2305) and dom.webnotifcations* (2304) are somehow linked - it's somewhere in the list of links in the 2300 section header, that one is obsolete or controlled by the other. I can't seem to find it.
At the end of the day - workers SUCK from a control and privacy perspective. They have their own permissions for IDB and their own caches and all sorts. Nightmare .. and a bit of a mystery (to me).
1
see next post

1

Setup: new profile in FF56
Note: dom.storage is default inactive in the user.js, so is assumed to be allowed - only really looking at the workers stuff
Note: only 1st party cookies allowed
Note: I am not blocking any JS or third party content (cept 3rd party cookies) - no NoScript, no uBo, no uMatrix
Note: images are coming from a 3rd party: akamaized

:small_orange_diamond: when I reset

close the site
clear everything via ctrl-shift-del (yes this clears the IDB entries)

:small_blue_diamond: test 1

load site, works fine
creates IDB entires for site and also creativecdn
creates a cookie for site

:small_blue_diamond: reset between tests & tests 2+3+4+5+6+7

each of these prefs were done in individual subsequent (accumulative) tests
test 2: set dom.workers.enabled => false

note: no IDB entries anymore - interesting that workers, like service workers, seems to allow IDB

test 3: set dom.serviceWorkers.enabled => false
test 4: set dom.webnotifications.enabled => false
test 5: set dom.webnotifications.serviceworker.enabled => false
test 6: load site and deny cookies via site exceptions & reset & test

I assume you would want cookies anyway if you were to login and buy something

test 7: set dom.storage.enabled => false
load site, works fine

Conclusion
Well I disabled everything (everything workers, local storage - in fact all storage including IDB) and it works for me. I even killed cookies on the site (and blocked all 3rd party cookies).
I do not think the issue is what you think it is - I cannot replicate the problem. I suggest you look at uBo, uM logger details for what happens with requests to akamaized (which serve the images)

^^ I also just applied the user.js to this profile on top of the above changes/tests (so dom.storage.enabled is still false), and it still works

Pants, you can not replicate the problem because the site has been updated. It's a pity, because we won't understand what dom.webnotifications.enabled has to do with image loading unless we find another site with the same issue.
I tested it properly 23 days ago. Brand new profile + user.js, no plugins, no port blocking, etc.

Pants, you can not replicate the problem because the site has been updated

Well .. poop!

It still requires dom.storage to be enabled otherwise categories and product browsing doesn't work (FF52).

Well, I clicked around, including product browsing (not sure if I got a category, but the menus on the left seem to indicate that's what they were - yes I read you original email about going into categories), and it seemed to work for me without dom.storage - but that's OK, we allow this in the user.js as the setting is inactive

It's a pity, because we won't understand what dom.webnotifications.enabled has to do with image loading unless we find another site with the same issue.

Maybe some more reading on the spec will give some insight into what this can do

@Just-me-ghacks
https://developer.mozilla.org/en-US/docs/Web/API/Notifications_API/Using_the_Notifications_API
I will assume from this, that since it can load/store data into IDB, then these resources become available for for the domain?

Maybe. However that means user_pref("dom.webnotifications.enabled", true); bypass user_pref("dom.indexedDB.enabled", false); which is strange.

However that means user_pref("dom.webnotifications.enabled", true); bypass user_pref("dom.indexedDB.enabled", false); which is strange

Did you test that? Or did I say that? I am getting confused. The whole storage thing is doing my head in and I've almost had enough of it :) . Like I said, seems workers have their own storage rights - eg see that YT issue where I was trying to work out how IDB entries were being created. It might become more clear when Storage Manager is finished
It's all getting too confusing TBH .. I wonder if cookies denied blocks IDB for workers? (probably not? but then PB mode denies IDB - so does IDB=false deny worker's ability to write to IDB?

Did you test that? Or did I say that?

No, no, it's just a (logical) assumption - I have indexedDB disabled by default. The site required dom storage and webnotifications to display images. So how can user_pref("dom.webnotifications.enabled", true); store data into indexedDB when indexedDB is disabled?
The site in question has been updated. The issue is not common, so I suggest we just close this thread until we find another site that loads images / content via webnotifications.

No, no, it's just a (logical) assumption. I have indexedDB disabled by default. The site required dom storage and webnotifications to display images. So how can user_pref("dom.webnotifications.enabled", true); store data into indexedDB when indexedDB is disabled?

Yup: assumption. my observation of IDB being used is from the technical docs example. In other words, it needs a test, and because the site code changed there is no way to tell if it actually used IDB
I'll leave this open to remind me to do a couple of test re workers & IDB

P.S.: the site has been restored! Right now it requires

Yeah. I guess that change was just for Halloween. I cannot replicate your results.
In a brand new FF profile

Default nilla: NOTE: There is a slideshow box with left/right arrows and 6 dots to indicate slideshow position on the HOME landing page - this is not filled in and remains blank, even in a nilla FF. It does fill with text when you click a category from the menu on the left.
I am looking at the images that load underneath that, and images in a category (tvs) and even images on a product (and the mouse over shows all images and the slideshow works if you pop that up)
no IDB was created

tests (reset in between each one)

set IDB=false = same result

set dom.workers.enabled => false = same result

set dom.serviceWorkers.enabled => fasle = same result

dom.webnotifications.enabled => false = same result

set dom.webnotifications.serviceworker.enabled => false = same result

denied the site a cookie and reset = same result
Conclusion: workers have nothing to do with the images being loaded

Lastly

set dom.storage.enabled = false = pics broken

I guess that change was just for Halloween.

Maybe they are A/B testing something.
Right now it requires only user_pref("dom.storage.enabled", true); to display images.
user_pref("dom.webnotifications.enabled", true); is not required.

closing this, fell free to reopen if more info comes to light


Publish the action also into GitHub Market place.

I don't see this action on GitHub Marketplace like testing-farm-as-github-action https://github.com/marketplace/actions/schedule-tests-on-testing-farm.
Can you please publish it?

Hi, I did not publish the action, because it is only used within SCLorg. But I can release it if you think, it is needed.

It is available now: https://github.com/marketplace/actions/build-and-push-images-to-registry


Add new support channel

Add gitter channel
inform that there are local forums other than stackexchange (especially for non-english people)
break lines to around 80 characters
Question: is freenode channel still active?

Question: is freenode channel still active?
Yes!

Does gitter official?

I don't know how and when a chat can be considered as official in QGIs project. I remembered there was a mail about gitter in dev list but at https://gitter.im/qgis/QGIS you'll see most of the dev of QGIS. So I suppose it's worth mentionning as a mean to get up to date info

Indeed, freenode is very active, even though imho there are many questions unanswered.

@DelazJ do you think this PR is ready to be merged?

wait, will ask about gitter on mailing list

@yjacolin I forgot to point that the question has been asked here: http://osgeo-org.1560.x6.nabble.com/Qgis-community-team-Is-gitter-an-official-mean-to-ask-about-QGIS-td5260602.html. Will wait a few days and if no answer...


test error on Rails 5.0.1

app/controllers/v1/base_controller.rb:
class V1::BaseController < ApplicationController
end

test/integration/v1/base_controller_test.rb:
require 'test_helper'

class V1::BaseControllerTest < ActionDispatch::IntegrationTest
  # Replace this with your real tests.
  test "the truth" do
    assert true
  end
end

Found it. There's alos a functional test class with the same class name in test/functional/v1/base_controller_test.rb:
require 'test_helper'

class V1::BaseControllerTest < ActionController::TestCase
  # Replace this with your real tests.
  test "the truth" do
    assert true
  end
end

@bploetz I used the versionist:new_api_version generator to generate the template code.  Should the functional test class name be different with the integration test?

versionist 1.6.0 has been released with the fix for this issue.


Question about the training strategy with frozen generative LLM

When the generative LLM is frozen, how do we make sure the output embedding of the R^2-former is meaningful to the generative LLM? In that case, do you actually replace the <R> placeholder with the actual score of the query-doc matching model? It's not quite clear from the original paper.

Thanks for your questions!

We utilize a joint training strategy to ensure the alignment between the R^2-Former and LLM.
The R^2-Former outputs an embedding, not a query-doc matching score.
In our implementation, we mark the <R> placeholder with a specific token id (such as 0 in LLaMA). We then replace its corresponding embedding with R^2-Former's output(modeling_rrag.py#L133-L135).

Thanks for the prompt reply! But in your paper, you did mentioned there was an option to freeze the LLM. How was it implemented??

Yes, the freeze_llm parameter control to freeze the LLM or not. modeling_rrag.py#L81-L84 shows the trigger.
Conversely, you can fine-tune the LLM by removing --freeze_llm in the script. Optionally, add --use_lora to enable PEFT (such as train_nq_lora.sh .sh).

Yes, the freeze_llm parameter control to freeze the LLM or not. modeling_rrag.py#L81-L84 shows the trigger. Conversely, you can fine-tune the LLM by removing --freeze_llm in the script. Optionally, add --use_lora to enable PEFT (such as train_nq_lora.sh .sh).

I meant, when you freeze the LLM, do you have a special strategy to deal with the <R> token? If R^2-Former is not aligned with the LLM, will it cause unexpected/undefined outcome?

The <R> token is just a placeholder, and it will be replaced before LLM processes the input embeddings. So, the LLM takes several embeddings as additional inputs. And, the training strategy ensures the additional embeddings from R^2-Former are meaningful for the LLM.
If the R^2-Former is not aligned, the LLM may be disturbed by these additional embeddings and generate some meaningless tokens.

Thanks again for the reply!

If the R^2-Former is not aligned, the LLM may be disturbed by these additional embeddings and generate some meaningless tokens

That's my understanding as well. But from the paper, I don't see how the R^2-Former model is aligned with LLM if the LLM is frozen (since the R^2-Former model is only trained on the QDM task). So I'm a bit confused why this works when LLM is frozen. Hope that clarifies my question :)

Oh, I got your point.
The key is the joint training, which involves LM and QDM tasks(not only QDM). The LM task makes the R^2-Former can receive feedback from the LLM(even frozen).
It is a common training strategy in multi-modality alignment, such as blip-2. You can regard the additional embeddings as another modal information for better understanding.

ah, I think I understand now. So when the LLM is frozen, I thought the LM loss is also removed. It makes sense to me now. Thanks a lot for the explanation!