Add support to read different neuron cache from the local directory of the DLC based on ENVs

Feature request
Under network isolation, SageMaker endpoint will not have access to the aws-neuron/optimum-neuron-cache to fetch cache.
Instead, we need pre-download the caches and model weights and send them as input for SageMaker endpoint deployment. During endpoint deployment, we expect the DLC read neuron caches from a local directory within DLC.
During previous communication with @dacorvo , below please find my minimal reproducible code.
import os
os.environ["HUGGINGFACE_HUB_CACHE"] = "<MY_LOCAL_DIR>"
!huggingface-cli download aws-neuron/optimum-neuron-cache --include "*neuronxcc-2.12.68*"
!huggingface-cli download meta-llama/Llama-2-7b-chat-hf

This will download all the neuron caches and llama-2 7b chat into MY_LOCAL_DIR, with directory structure as below.
-- models--aws-neuron--optimum-neuron-cache
-- models--meta-llama--Llama-2-7b-chat-hf

Then by using optimum-cli neuron cache lookup meta-llama/Llama-2-7b-hf I identified there is a neuron cache compiled for sequence length of 4096, batch size of 1, and neuron cores of 2, with neuronx-cc <IP_ADDRESS>+4480452af
Further I upload MY_LOCAL_DIR into S3 bucket say MY_S3, and use following SDK code to deploy an endpoint.
import json
import sagemaker
import boto3
from sagemaker.huggingface import HuggingFaceModel, get_huggingface_llm_image_uri
from sagemaker.model import Model

try:
    role = sagemaker.get_execution_role()
except ValueError:
    iam = boto3.client("iam")
    role = iam.get_role(RoleName="sagemaker_execution_role")["Role"]["Arn"]
sagemaker_session = sagemaker.Session(boto_session=boto3.Session(region_name="us-east-1"))
endpoint_name = "llama-2-7b"

model_data = {
    'S3DataSource': {
        'CompressionType': 'None',
        'S3DataType': 'S3Prefix',
        'S3Uri': "<MY_S3>",
    }
}

# Hub Model configuration. https://huggingface.co/models
hub = {
    "HF_MODEL_ID": "meta-llama/Llama-2-7b-chat-hf",
    "HF_NUM_CORES": "2", # number of neuron cores
    "HF_BATCH_SIZE": "4", # batch size used to compile the model
    "HF_SEQUENCE_LENGTH": "4096", # length used to compile the model
    "HF_AUTO_CAST_TYPE": "fp16",  # dtype of the model
    "MAX_BATCH_SIZE": "4", # max batch size for the model
    "MAX_INPUT_LENGTH": "2048", # max length of input text
    "MAX_TOTAL_TOKENS": "4096", # max length of generated text
    #"HF_TOKEN": HfFolder.get_token(), # pass the huggingface token
    "HUGGINGFACE_HUB_CACHE": "/opt/ml/model", # What David suggested
    "TRANSFORMERS_CACHE": "/tmp"
}

model = Model(
    image_uri="763104351884.dkr.ecr.us-east-1.amazonaws.com/huggingface-pytorch-tgi-inference:1.13.1-optimum0.0.20-neuronx-py310-ubuntu22.04",
    model_data=model_data,
    role=role,
    sagemaker_session=sagemaker_session,
    name=endpoint_name,
    env=hub,
)

# deploy model to SageMaker Inference
predictor = model.deploy(
    initial_instance_count=1,
    instance_type="ml.inf2.8xlarge",
    endpoint_name=endpoint_name,
    volume_size=512,
    model_data_download_timeout=3600,
    container_startup_health_check_timeout=3600,
)

# send request
predictor.predict(
    {
        "inputs": "What is is the capital of France?",
    }
)

Error occurs during deployment and is shown as below. The error seems from trying to access a gated llama model and access denied. But with model weights included in the local dir of DLC, it shouldn't go to public internet to fetch it.
2024-03-28T01:20:31.908Z
#033[2m2024-03-28T01:20:31.082918Z#033[0m #033[32m INFO#033[0m #033[2mtext_generation_launcher#033[0m#033[2m:#033[0m Args { model_id: "meta-llama/Llama-2-7b-chat-hf", revision: None, validation_workers: 2, sharded: None, num_shard: None, quantize: None, speculate: None, dtype: None, trust_remote_code: false, max_concurrent_requests: 128, max_best_of: 2, max_stop_sequences: 4, max_top_n_tokens: 5, max_input_length: 2048, max_total_tokens: 4096, waiting_served_ratio: 1.2, max_batch_prefill_tokens: 4096, max_batch_total_tokens: None, max_waiting_tokens: 20, max_batch_size: Some(4), enable_cuda_graphs: false, hostname: "container-0.local", port: 8080, shard_uds_path: "/tmp/text-generation-server", master_addr: "localhost", master_port: 29500, huggingface_hub_cache: Some("/opt/ml/model"), weights_cache_override: None, disable_custom_kernels: false, cuda_memory_fraction: 1.0, rope_scaling: None, rope_factor: None, json_output: false, otlp_endpoint: None, cors_allow_origin: [], watermark_gamma: None, watermark_delta: None, ngrok: false, ngrok_authtoken: None, ngrok_edge: None, tokenizer_config_path: None, disable_grammar_support: false, env: false }

Copy
#033[2m2024-03-28T01:20:31.082918Z#033[0m #033[32m INFO#033[0m #033[2mtext_generation_launcher#033[0m#033[2m:#033[0m Args { model_id: "meta-llama/Llama-2-7b-chat-hf", revision: None, validation_workers: 2, sharded: None, num_shard: None, quantize: None, speculate: None, dtype: None, trust_remote_code: false, max_concurrent_requests: 128, max_best_of: 2, max_stop_sequences: 4, max_top_n_tokens: 5, max_input_length: 2048, max_total_tokens: 4096, waiting_served_ratio: 1.2, max_batch_prefill_tokens: 4096, max_batch_total_tokens: None, max_waiting_tokens: 20, max_batch_size: Some(4), enable_cuda_graphs: false, hostname: "container-0.local", port: 8080, shard_uds_path: "/tmp/text-generation-server", master_addr: "localhost", master_port: 29500, huggingface_hub_cache: Some("/opt/ml/model"), weights_cache_override: None, disable_custom_kernels: false, cuda_memory_fraction: 1.0, rope_scaling: None, rope_factor: None, json_output: false, otlp_endpoint: None, cors_allow_origin: [], watermark_gamma: None, watermark_delta: None, ngrok: false, ngrok_authtoken: None, ngrok_edge: None, tokenizer_config_path: None, disable_grammar_support: false, env: false }
2024-03-28T01:20:31.908Z
#033[2m2024-03-28T01:20:31.082998Z#033[0m #033[32m INFO#033[0m #033[1mdownload#033[0m: #033[2mtext_generation_launcher#033[0m#033[2m:#033[0m Starting download process.

Copy
#033[2m2024-03-28T01:20:31.082998Z#033[0m #033[32m INFO#033[0m #033[1mdownload#033[0m: #033[2mtext_generation_launcher#033[0m#033[2m:#033[0m Starting download process.
2024-03-28T01:20:32.907Z
#033[2m2024-03-28T01:20:31.174030Z#033[0m #033[33m WARN#033[0m #033[2mtext_generation_launcher#033[0m#033[2m:#033[0m 'extension' argument is not supported and will be ignored.

Copy
#033[2m2024-03-28T01:20:31.174030Z#033[0m #033[33m WARN#033[0m #033[2mtext_generation_launcher#033[0m#033[2m:#033[0m 'extension' argument is not supported and will be ignored.
2024-03-28T01:20:32.907Z
#033[2m2024-03-28T01:20:32.885603Z#033[0m #033[31mERROR#033[0m #033[1mdownload#033[0m: #033[2mtext_generation_launcher#033[0m#033[2m:#033[0m Download encountered an error:

Copy
#033[2m2024-03-28T01:20:32.885603Z#033[0m #033[31mERROR#033[0m #033[1mdownload#033[0m: #033[2mtext_generation_launcher#033[0m#033[2m:#033[0m Download encountered an error:
2024-03-28T01:20:32.907Z
/usr/local/lib/python3.10/dist-packages/transformers/utils/hub.py:123: FutureWarning: Using `TRANSFORMERS_CACHE` is deprecated and will be removed in v5 of Transformers. Use `HF_HOME` instead.
  warnings.warn(

Copy
/usr/local/lib/python3.10/dist-packages/transformers/utils/hub.py:123: FutureWarning: Using `TRANSFORMERS_CACHE` is deprecated and will be removed in v5 of Transformers. Use `HF_HOME` instead. warnings.warn(
2024-03-28T01:20:32.907Z
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_errors.py", line 304, in hf_raise_for_status
    response.raise_for_status()
  File "/usr/local/lib/python3.10/dist-packages/requests/models.py", line 1021, in raise_for_status

Copy
Traceback (most recent call last): File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_errors.py", line 304, in hf_raise_for_status response.raise_for_status() File "/usr/local/lib/python3.10/dist-packages/requests/models.py", line 1021, in raise_for_status
2024-03-28T01:20:32.907Z
Error: DownloadError
    raise HTTPError(http_error_msg, response=self)

Copy
Error: DownloadError raise HTTPError(http_error_msg, response=self)
2024-03-28T01:20:32.907Z
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://huggingface.co/meta-llama/Llama-2-7b-chat-hf/resolve/main/config.json

Copy
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://huggingface.co/meta-llama/Llama-2-7b-chat-hf/resolve/main/config.json
2024-03-28T01:20:32.907Z
The above exception was the direct cause of the following exception:

Copy
The above exception was the direct cause of the following exception:
2024-03-28T01:20:32.907Z
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/transformers/utils/hub.py", line 389, in cached_file
    resolved_file = hf_hub_download(
  File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_validators.py", line 118, in _inner_fn
    return fn(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py", line 1403, in hf_hub_download
    raise head_call_error
  File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py", line 1261, in hf_hub_download
    metadata = get_hf_file_metadata(
  File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_validators.py", line 118, in _inner_fn
    return fn(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py", line 1667, in get_hf_file_metadata
    r = _request_wrapper(
  File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py", line 385, in _request_wrapper
    response = _request_wrapper(
  File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py", line 409, in _request_wrapper
    hf_raise_for_status(response)
  File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_errors.py", line 321, in hf_raise_for_status
    raise GatedRepoError(message, response) from e

Copy
Traceback (most recent call last): File "/usr/local/lib/python3.10/dist-packages/transformers/utils/hub.py", line 389, in cached_file resolved_file = hf_hub_download( File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_validators.py", line 118, in _inner_fn return fn(*args, **kwargs) File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py", line 1403, in hf_hub_download raise head_call_error File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py", line 1261, in hf_hub_download metadata = get_hf_file_metadata( File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_validators.py", line 118, in _inner_fn return fn(*args, **kwargs) File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py", line 1667, in get_hf_file_metadata r = _request_wrapper( File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py", line 385, in _request_wrapper response = _request_wrapper( File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py", line 409, in _request_wrapper hf_raise_for_status(response) File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_errors.py", line 321, in hf_raise_for_status raise GatedRepoError(message, response) from e
2024-03-28T01:20:32.907Z
huggingface_hub.utils._errors.GatedRepoError: 401 Client Error. (Request ID: Root=1-6604c5e0-374ff03938f405767aab5d5d;00596397-d2ea-432e-a5cd-d0a0661b53e7)

Copy
huggingface_hub.utils._errors.GatedRepoError: 401 Client Error. (Request ID: Root=1-6604c5e0-374ff03938f405767aab5d5d;00596397-d2ea-432e-a5cd-d0a0661b53e7)
2024-03-28T01:20:32.907Z
Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-2-7b-chat-hf/resolve/main/config.json.

Copy
Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-2-7b-chat-hf/resolve/main/config.json.
2024-03-28T01:20:32.907Z
Repo model meta-llama/Llama-2-7b-chat-hf is gated. You must be authenticated to access it.

Copy
Repo model meta-llama/Llama-2-7b-chat-hf is gated. You must be authenticated to access it.
2024-03-28T01:20:32.907Z
The above exception was the direct cause of the following exception:

Copy
The above exception was the direct cause of the following exception:
2024-03-28T01:20:32.907Z
Traceback (most recent call last):
  File "/usr/local/bin/text-generation-server", line 8, in <module>
    sys.exit(app())
  File "/usr/local/lib/python3.10/dist-packages/text_generation_server/cli.py", line 101, in download_weights
    fetch_model(model_id, revision)
  File "/usr/local/lib/python3.10/dist-packages/text_generation_server/model.py", line 87, in fetch_model
    config = AutoConfig.from_pretrained(model_id, revision=revision)
  File "/usr/local/lib/python3.10/dist-packages/transformers/models/auto/configuration_auto.py", line 1082, in from_pretrained
    config_dict, unused_kwargs = PretrainedConfig.get_config_dict(pretrained_model_name_or_path, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/transformers/configuration_utils.py", line 644, in get_config_dict
    config_dict, kwargs = cls._get_config_dict(pretrained_model_name_or_path, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/transformers/configuration_utils.py", line 699, in _get_config_dict
    resolved_config_file = cached_file(
  File "/usr/local/lib/python3.10/dist-packages/transformers/utils/hub.py", line 404, in cached_file
    raise EnvironmentError(

Copy
Traceback (most recent call last): File "/usr/local/bin/text-generation-server", line 8, in <module> sys.exit(app()) File "/usr/local/lib/python3.10/dist-packages/text_generation_server/cli.py", line 101, in download_weights fetch_model(model_id, revision) File "/usr/local/lib/python3.10/dist-packages/text_generation_server/model.py", line 87, in fetch_model config = AutoConfig.from_pretrained(model_id, revision=revision) File "/usr/local/lib/python3.10/dist-packages/transformers/models/auto/configuration_auto.py", line 1082, in from_pretrained config_dict, unused_kwargs = PretrainedConfig.get_config_dict(pretrained_model_name_or_path, **kwargs) File "/usr/local/lib/python3.10/dist-packages/transformers/configuration_utils.py", line 644, in get_config_dict config_dict, kwargs = cls._get_config_dict(pretrained_model_name_or_path, **kwargs) File "/usr/local/lib/python3.10/dist-packages/transformers/configuration_utils.py", line 699, in _get_config_dict resolved_config_file = cached_file( File "/usr/local/lib/python3.10/dist-packages/transformers/utils/hub.py", line 404, in cached_file raise EnvironmentError(
2024-03-28T01:20:32.907Z
OSError: You are trying to access a gated repo.

Copy
OSError: You are trying to access a gated repo.
2024-03-28T01:20:33.157Z
Make sure to request access at https://huggingface.co/meta-llama/Llama-2-7b-chat-hf and pass a token having permission to this repo either by logging in with `huggingface-cli login` or by passing `token=<your_token>`.

Copy
Make sure to request access at https://huggingface.co/meta-llama/Llama-2-7b-chat-hf and pass a token having permission to this repo either by logging in with `huggingface-cli login` or by passing `token=<your_token>`.
2024-03-28T01:20:33.157Z
#033[2m2024-03-28T01:20:33.048196Z#033[0m #033[32m INFO#033[0m #033[2mtext_generation_launcher#033[0m#033[2m:#033[0m Args { model_id: "meta-llama/Llama-2-7b-chat-hf", revision: None, validation_workers: 2, sharded: None, num_shard: None, quantize: None, speculate: None, dtype: None, trust_remote_code: false, max_concurrent_requests: 128, max_best_of: 2, max_stop_sequences: 4, max_top_n_tokens: 5, max_input_length: 2048, max_total_tokens: 4096, waiting_served_ratio: 1.2, max_batch_prefill_tokens: 4096, max_batch_total_tokens: None, max_waiting_tokens: 20, max_batch_size: Some(4), enable_cuda_graphs: false, hostname: "container-0.local", port: 8080, shard_uds_path: "/tmp/text-generation-server", master_addr: "localhost", master_port: 29500, huggingface_hub_cache: Some("/opt/ml/model"), weights_cache_override: None, disable_custom_kernels: false, cuda_memory_fraction: 1.0, rope_scaling: None, rope_factor: None, json_output: false, otlp_endpoint: None, cors_allow_origin: [], watermark_gamma: None, watermark_delta: None, ngrok: false, ngrok_authtoken: None, ngrok_edge: None, tokenizer_config_path: None, disable_grammar_support: false, env: false }

Copy
#033[2m2024-03-28T01:20:33.048196Z#033[0m #033[32m INFO#033[0m #033[2mtext_generation_launcher#033[0m#033[2m:#033[0m Args { model_id: "meta-llama/Llama-2-7b-chat-hf", revision: None, validation_workers: 2, sharded: None, num_shard: None, quantize: None, speculate: None, dtype: None, trust_remote_code: false, max_concurrent_requests: 128, max_best_of: 2, max_stop_sequences: 4, max_top_n_tokens: 5, max_input_length: 2048, max_total_tokens: 4096, waiting_served_ratio: 1.2, max_batch_prefill_tokens: 4096, max_batch_total_tokens: None, max_waiting_tokens: 20, max_batch_size: Some(4), enable_cuda_graphs: false, hostname: "container-0.local", port: 8080, shard_uds_path: "/tmp/text-generation-server", master_addr: "localhost", master_port: 29500, huggingface_hub_cache: Some("/opt/ml/model"), weights_cache_override: None, disable_custom_kernels: false, cuda_memory_fraction: 1.0, rope_scaling: None, rope_factor: None, json_output: false, otlp_endpoint: None, cors_allow_origin: [], watermark_gamma: None, watermark_delta: None, ngrok: false, ngrok_authtoken: None, ngrok_edge: None, tokenizer_config_path: None, disable_grammar_support: false, env: false }
2024-03-28T01:20:33.157Z
#033[2m2024-03-28T01:20:33.048279Z#033[0m #033[32m INFO#033[0m #033[1mdownload#033[0m: #033[2mtext_generation_launcher#033[0m#033[2m:#033[0m Starting download process.

Copy
#033[2m2024-03-28T01:20:33.048279Z#033[0m #033[32m INFO#033[0m #033[1mdownload#033[0m: #033[2mtext_generation_launcher#033[0m#033[2m:#033[0m Starting download process.
2024-03-28T01:20:34.911Z
#033[2m2024-03-28T01:20:33.139764Z#033[0m #033[33m WARN#033[0m #033[2mtext_generation_launcher#033[0m#033[2m:#033[0m 'extension' argument is not supported and will be ignored.

Copy
#033[2m2024-03-28T01:20:33.139764Z#033[0m #033[33m WARN#033[0m #033[2mtext_generation_launcher#033[0m#033[2m:#033[0m 'extension' argument is not supported and will be ignored.
2024-03-28T01:20:34.911Z
#033[2m2024-03-28T01:20:34.850506Z#033[0m #033[31mERROR#033[0m #033[1mdownload#033[0m: #033[2mtext_generation_launcher#033[0m#033[2m:#033[0m Download encountered an error:

Copy
#033[2m2024-03-28T01:20:34.850506Z#033[0m #033[31mERROR#033[0m #033[1mdownload#033[0m: #033[2mtext_generation_launcher#033[0m#033[2m:#033[0m Download encountered an error:
2024-03-28T01:20:34.912Z
/usr/local/lib/python3.10/dist-packages/transformers/utils/hub.py:123: FutureWarning: Using `TRANSFORMERS_CACHE` is deprecated and will be removed in v5 of Transformers. Use `HF_HOME` instead.
  warnings.warn(

Copy
/usr/local/lib/python3.10/dist-packages/transformers/utils/hub.py:123: FutureWarning: Using `TRANSFORMERS_CACHE` is deprecated and will be removed in v5 of Transformers. Use `HF_HOME` instead. warnings.warn(
2024-03-28T01:20:34.912Z
Traceback (most recent call last):

Copy
Traceback (most recent call last):
2024-03-28T01:20:34.912Z
Error: DownloadError
  File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_errors.py", line 304, in hf_raise_for_status
    response.raise_for_status()
  File "/usr/local/lib/python3.10/dist-packages/requests/models.py", line 1021, in raise_for_status
    raise HTTPError(http_error_msg, response=self)

Copy
Error: DownloadError File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_errors.py", line 304, in hf_raise_for_status response.raise_for_status() File "/usr/local/lib/python3.10/dist-packages/requests/models.py", line 1021, in raise_for_status raise HTTPError(http_error_msg, response=self)
2024-03-28T01:20:34.912Z
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://huggingface.co/meta-llama/Llama-2-7b-chat-hf/resolve/main/config.json

Copy
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://huggingface.co/meta-llama/Llama-2-7b-chat-hf/resolve/main/config.json
2024-03-28T01:20:34.912Z
The above exception was the direct cause of the following exception:

Copy
The above exception was the direct cause of the following exception:
2024-03-28T01:20:34.912Z
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/transformers/utils/hub.py", line 389, in cached_file
    resolved_file = hf_hub_download(
  File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_validators.py", line 118, in _inner_fn
    return fn(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py", line 1403, in hf_hub_download
    raise head_call_error
  File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py", line 1261, in hf_hub_download
    metadata = get_hf_file_metadata(
  File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_validators.py", line 118, in _inner_fn
    return fn(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py", line 1667, in get_hf_file_metadata
    r = _request_wrapper(
  File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py", line 385, in _request_wrapper
    response = _request_wrapper(
  File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py", line 409, in _request_wrapper
    hf_raise_for_status(response)
  File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_errors.py", line 321, in hf_raise_for_status
    raise GatedRepoError(message, response) from e

Copy
Traceback (most recent call last): File "/usr/local/lib/python3.10/dist-packages/transformers/utils/hub.py", line 389, in cached_file resolved_file = hf_hub_download( File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_validators.py", line 118, in _inner_fn return fn(*args, **kwargs) File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py", line 1403, in hf_hub_download raise head_call_error File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py", line 1261, in hf_hub_download metadata = get_hf_file_metadata( File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_validators.py", line 118, in _inner_fn return fn(*args, **kwargs) File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py", line 1667, in get_hf_file_metadata r = _request_wrapper( File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py", line 385, in _request_wrapper response = _request_wrapper( File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py", line 409, in _request_wrapper hf_raise_for_status(response) File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_errors.py", line 321, in hf_raise_for_status raise GatedRepoError(message, response) from e
2024-03-28T01:20:34.912Z
huggingface_hub.utils._errors.GatedRepoError: 401 Client Error. (Request ID: Root=1-6604c5e2-70a6326a6a7d12926d9f955b;b5bb2db4-fbed-4b3f-b67c-8da67d35b816)

Copy
huggingface_hub.utils._errors.GatedRepoError: 401 Client Error. (Request ID: Root=1-6604c5e2-70a6326a6a7d12926d9f955b;b5bb2db4-fbed-4b3f-b67c-8da67d35b816)
2024-03-28T01:20:34.912Z
Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-2-7b-chat-hf/resolve/main/config.json.

Copy
Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-2-7b-chat-hf/resolve/main/config.json.
2024-03-28T01:20:34.912Z
Repo model meta-llama/Llama-2-7b-chat-hf is gated. You must be authenticated to access it.

Copy
Repo model meta-llama/Llama-2-7b-chat-hf is gated. You must be authenticated to access it.
2024-03-28T01:20:34.912Z
The above exception was the direct cause of the following exception:

Copy
The above exception was the direct cause of the following exception:
2024-03-28T01:20:34.912Z
Traceback (most recent call last):
  File "/usr/local/bin/text-generation-server", line 8, in <module>
    sys.exit(app())
  File "/usr/local/lib/python3.10/dist-packages/text_generation_server/cli.py", line 101, in download_weights
    fetch_model(model_id, revision)
  File "/usr/local/lib/python3.10/dist-packages/text_generation_server/model.py", line 87, in fetch_model
    config = AutoConfig.from_pretrained(model_id, revision=revision)
  File "/usr/local/lib/python3.10/dist-packages/transformers/models/auto/configuration_auto.py", line 1082, in from_pretrained
    config_dict, unused_kwargs = PretrainedConfig.get_config_dict(pretrained_model_name_or_path, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/transformers/configuration_utils.py", line 644, in get_config_dict
    config_dict, kwargs = cls._get_config_dict(pretrained_model_name_or_path, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/transformers/configuration_utils.py", line 699, in _get_config_dict
    resolved_config_file = cached_file(
  File "/usr/local/lib/python3.10/dist-packages/transformers/utils/hub.py", line 404, in cached_file
    raise EnvironmentError(

Copy
Traceback (most recent call last): File "/usr/local/bin/text-generation-server", line 8, in <module> sys.exit(app()) File "/usr/local/lib/python3.10/dist-packages/text_generation_server/cli.py", line 101, in download_weights fetch_model(model_id, revision) File "/usr/local/lib/python3.10/dist-packages/text_generation_server/model.py", line 87, in fetch_model config = AutoConfig.from_pretrained(model_id, revision=revision) File "/usr/local/lib/python3.10/dist-packages/transformers/models/auto/configuration_auto.py", line 1082, in from_pretrained config_dict, unused_kwargs = PretrainedConfig.get_config_dict(pretrained_model_name_or_path, **kwargs) File "/usr/local/lib/python3.10/dist-packages/transformers/configuration_utils.py", line 644, in get_config_dict config_dict, kwargs = cls._get_config_dict(pretrained_model_name_or_path, **kwargs) File "/usr/local/lib/python3.10/dist-packages/transformers/configuration_utils.py", line 699, in _get_config_dict resolved_config_file = cached_file( File "/usr/local/lib/python3.10/dist-packages/transformers/utils/hub.py", line 404, in cached_file raise EnvironmentError(
2024-03-28T01:20:34.912Z
OSError: You are trying to access a gated repo.

Copy
OSError: You are trying to access a gated repo.
2024-03-28T01:20:35.162Z
Make sure to request access at https://huggingface.co/meta-llama/Llama-2-7b-chat-hf and pass a token having permission to this repo either by logging in with `huggingface-cli login` or by passing `token=<your_token>`.

Copy
Make sure to request access at https://huggingface.co/meta-llama/Llama-2-7b-chat-hf and pass a token having permission to this repo either by logging in with `huggingface-cli login` or by passing `token=<your_token>`.
2024-03-28T01:20:35.162Z
#033[2m2024-03-28T01:20:35.111808Z#033[0m #033[32m INFO#033[0m #033[2mtext_generation_launcher#033[0m#033[2m:#033[0m Args { model_id: "meta-llama/Llama-2-7b-chat-hf", revision: None, validation_workers: 2, sharded: None, num_shard: None, quantize: None, speculate: None, dtype: None, trust_remote_code: false, max_concurrent_requests: 128, max_best_of: 2, max_stop_sequences: 4, max_top_n_tokens: 5, max_input_length: 2048, max_total_tokens: 4096, waiting_served_ratio: 1.2, max_batch_prefill_tokens: 4096, max_batch_total_tokens: None, max_waiting_tokens: 20, max_batch_size: Some(4), enable_cuda_graphs: false, hostname: "container-0.local", port: 8080, shard_uds_path: "/tmp/text-generation-server", master_addr: "localhost", master_port: 29500, huggingface_hub_cache: Some("/opt/ml/model"), weights_cache_override: None, disable_custom_kernels: false, cuda_memory_fraction: 1.0, rope_scaling: None, rope_factor: None, json_output: false, otlp_endpoint: None, cors_allow_origin: [], watermark_gamma: None, watermark_delta: None, ngrok: false, ngrok_authtoken: None, ngrok_edge: None, tokenizer_config_path: None, disable_grammar_support: false, env: false }

Copy
#033[2m2024-03-28T01:20:35.111808Z#033[0m #033[32m INFO#033[0m #033[2mtext_generation_launcher#033[0m#033[2m:#033[0m Args { model_id: "meta-llama/Llama-2-7b-chat-hf", revision: None, validation_workers: 2, sharded: None, num_shard: None, quantize: None, speculate: None, dtype: None, trust_remote_code: false, max_concurrent_requests: 128, max_best_of: 2, max_stop_sequences: 4, max_top_n_tokens: 5, max_input_length: 2048, max_total_tokens: 4096, waiting_served_ratio: 1.2, max_batch_prefill_tokens: 4096, max_batch_total_tokens: None, max_waiting_tokens: 20, max_batch_size: Some(4), enable_cuda_graphs: false, hostname: "container-0.local", port: 8080, shard_uds_path: "/tmp/text-generation-server", master_addr: "localhost", master_port: 29500, huggingface_hub_cache: Some("/opt/ml/model"), weights_cache_override: None, disable_custom_kernels: false, cuda_memory_fraction: 1.0, rope_scaling: None, rope_factor: None, json_output: false, otlp_endpoint: None, cors_allow_origin: [], watermark_gamma: None, watermark_delta: None, ngrok: false, ngrok_authtoken: None, ngrok_edge: None, tokenizer_config_path: None, disable_grammar_support: false, env: false }
2024-03-28T01:20:35.412Z
#033[2m2024-03-28T01:20:35.111885Z#033[0m #033[32m INFO#033[0m #033[1mdownload#033[0m: #033[2mtext_generation_launcher#033[0m#033[2m:#033[0m Starting download process.

Copy
#033[2m2024-03-28T01:20:35.111885Z#033[0m #033[32m INFO#033[0m #033[1mdownload#033[0m: #033[2mtext_generation_launcher#033[0m#033[2m:#033[0m Starting download process.
2024-03-28T01:20:37.180Z
#033[2m2024-03-28T01:20:35.202901Z#033[0m #033[33m WARN#033[0m #033[2mtext_generation_launcher#033[0m#033[2m:#033[0m 'extension' argument is not supported and will be ignored.

Copy
#033[2m2024-03-28T01:20:35.202901Z#033[0m #033[33m WARN#033[0m #033[2mtext_generation_launcher#033[0m#033[2m:#033[0m 'extension' argument is not supported and will be ignored.
2024-03-28T01:20:37.180Z
#033[2m2024-03-28T01:20:36.914374Z#033[0m #033[31mERROR#033[0m #033[1mdownload#033[0m: #033[2mtext_generation_launcher#033[0m#033[2m:#033[0m Download encountered an error:

Copy
#033[2m2024-03-28T01:20:36.914374Z#033[0m #033[31mERROR#033[0m #033[1mdownload#033[0m: #033[2mtext_generation_launcher#033[0m#033[2m:#033[0m Download encountered an error:
2024-03-28T01:20:37.180Z
/usr/local/lib/python3.10/dist-packages/transformers/utils/hub.py:123: FutureWarning: Using `TRANSFORMERS_CACHE` is deprecated and will be removed in v5 of Transformers. Use `HF_HOME` instead.
  warnings.warn(

Copy
/usr/local/lib/python3.10/dist-packages/transformers/utils/hub.py:123: FutureWarning: Using `TRANSFORMERS_CACHE` is deprecated and will be removed in v5 of Transformers. Use `HF_HOME` instead. warnings.warn(
2024-03-28T01:20:37.180Z
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_errors.py", line 304, in hf_raise_for_status
    response.raise_for_status()
  File "/usr/local/lib/python3.10/dist-packages/requests/models.py", line 1021, in raise_for_status
    raise HTTPError(http_error_msg, response=self)

Copy
Traceback (most recent call last): File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_errors.py", line 304, in hf_raise_for_status response.raise_for_status() File "/usr/local/lib/python3.10/dist-packages/requests/models.py", line 1021, in raise_for_status raise HTTPError(http_error_msg, response=self)
2024-03-28T01:20:37.180Z
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://huggingface.co/meta-llama/Llama-2-7b-chat-hf/resolve/main/config.json

Copy
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://huggingface.co/meta-llama/Llama-2-7b-chat-hf/resolve/main/config.json
2024-03-28T01:20:37.180Z
The above exception was the direct cause of the following exception:

Copy
The above exception was the direct cause of the following exception:
2024-03-28T01:20:37.180Z
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/transformers/utils/hub.py", line 389, in cached_file
    resolved_file = hf_hub_download(
  File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_validators.py", line 118, in _inner_fn

Copy
Traceback (most recent call last): File "/usr/local/lib/python3.10/dist-packages/transformers/utils/hub.py", line 389, in cached_file resolved_file = hf_hub_download( File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_validators.py", line 118, in _inner_fn
2024-03-28T01:20:37.180Z
Error: DownloadError
    return fn(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py", line 1403, in hf_hub_download
    raise head_call_error
  File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py", line 1261, in hf_hub_download
    metadata = get_hf_file_metadata(
  File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_validators.py", line 118, in _inner_fn
    return fn(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py", line 1667, in get_hf_file_metadata
    r = _request_wrapper(
  File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py", line 385, in _request_wrapper
    response = _request_wrapper(
  File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py", line 409, in _request_wrapper
    hf_raise_for_status(response)
  File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_errors.py", line 321, in hf_raise_for_status
    raise GatedRepoError(message, response) from e

Copy
Error: DownloadError return fn(*args, **kwargs) File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py", line 1403, in hf_hub_download raise head_call_error File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py", line 1261, in hf_hub_download metadata = get_hf_file_metadata( File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_validators.py", line 118, in _inner_fn return fn(*args, **kwargs) File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py", line 1667, in get_hf_file_metadata r = _request_wrapper( File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py", line 385, in _request_wrapper response = _request_wrapper( File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py", line 409, in _request_wrapper hf_raise_for_status(response) File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_errors.py", line 321, in hf_raise_for_status raise GatedRepoError(message, response) from e
2024-03-28T01:20:37.180Z
huggingface_hub.utils._errors.GatedRepoError: 401 Client Error. (Request ID: Root=1-6604c5e4-246b123f12b9734114e96ae5;6400278b-c5c5-4ed2-a036-54d8a69c5674)

Copy
huggingface_hub.utils._errors.GatedRepoError: 401 Client Error. (Request ID: Root=1-6604c5e4-246b123f12b9734114e96ae5;6400278b-c5c5-4ed2-a036-54d8a69c5674)
2024-03-28T01:20:37.180Z
Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-2-7b-chat-hf/resolve/main/config.json.

Copy
Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-2-7b-chat-hf/resolve/main/config.json.
2024-03-28T01:20:37.180Z
Repo model meta-llama/Llama-2-7b-chat-hf is gated. You must be authenticated to access it.

Copy
Repo model meta-llama/Llama-2-7b-chat-hf is gated. You must be authenticated to access it.
2024-03-28T01:20:37.180Z
The above exception was the direct cause of the following exception:

Copy
The above exception was the direct cause of the following exception:
2024-03-28T01:20:37.180Z
Traceback (most recent call last):
  File "/usr/local/bin/text-generation-server", line 8, in <module>
    sys.exit(app())
  File "/usr/local/lib/python3.10/dist-packages/text_generation_server/cli.py", line 101, in download_weights
    fetch_model(model_id, revision)
  File "/usr/local/lib/python3.10/dist-packages/text_generation_server/model.py", line 87, in fetch_model
    config = AutoConfig.from_pretrained(model_id, revision=revision)
  File "/usr/local/lib/python3.10/dist-packages/transformers/models/auto/configuration_auto.py", line 1082, in from_pretrained
    config_dict, unused_kwargs = PretrainedConfig.get_config_dict(pretrained_model_name_or_path, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/transformers/configuration_utils.py", line 644, in get_config_dict
    config_dict, kwargs = cls._get_config_dict(pretrained_model_name_or_path, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/transformers/configuration_utils.py", line 699, in _get_config_dict
    resolved_config_file = cached_file(
  File "/usr/local/lib/python3.10/dist-packages/transformers/utils/hub.py", line 404, in cached_file
    raise EnvironmentError(

Copy
Traceback (most recent call last): File "/usr/local/bin/text-generation-server", line 8, in <module> sys.exit(app()) File "/usr/local/lib/python3.10/dist-packages/text_generation_server/cli.py", line 101, in download_weights fetch_model(model_id, revision) File "/usr/local/lib/python3.10/dist-packages/text_generation_server/model.py", line 87, in fetch_model config = AutoConfig.from_pretrained(model_id, revision=revision) File "/usr/local/lib/python3.10/dist-packages/transformers/models/auto/configuration_auto.py", line 1082, in from_pretrained config_dict, unused_kwargs = PretrainedConfig.get_config_dict(pretrained_model_name_or_path, **kwargs) File "/usr/local/lib/python3.10/dist-packages/transformers/configuration_utils.py", line 644, in get_config_dict config_dict, kwargs = cls._get_config_dict(pretrained_model_name_or_path, **kwargs) File "/usr/local/lib/python3.10/dist-packages/transformers/configuration_utils.py", line 699, in _get_config_dict resolved_config_file = cached_file( File "/usr/local/lib/python3.10/dist-packages/transformers/utils/hub.py", line 404, in cached_file raise EnvironmentError(
2024-03-28T01:20:37.180Z
OSError: You are trying to access a gated repo.

Copy
OSError: You are trying to access a gated repo.
2024-03-28T01:20:37.667Z
Make sure to request access at https://huggingface.co/meta-llama/Llama-2-7b-chat-hf and pass a token having permission to this repo either by logging in with `huggingface-cli login` or by passing `token=<your_token>`.

Copy
Make sure to request access at https://huggingface.co/meta-llama/Llama-2-7b-chat-hf and pass a token having permission to this repo either by logging in with `huggingface-cli login` or by passing `token=<your_token>`.
2024-03-28T01:20:37.667Z
#033[2m2024-03-28T01:20:37.609995Z#033[0m #033[32m INFO#033[0m #033[2mtext_generation_launcher#033[0m#033[2m:#033[0m Args { model_id: "meta-llama/Llama-2-7b-chat-hf", revision: None, validation_workers: 2, sharded: None, num_shard: None, quantize: None, speculate: None, dtype: None, trust_remote_code: false, max_concurrent_requests: 128, max_best_of: 2, max_stop_sequences: 4, max_top_n_tokens: 5, max_input_length: 2048, max_total_tokens: 4096, waiting_served_ratio: 1.2, max_batch_prefill_tokens: 4096, max_batch_total_tokens: None, max_waiting_tokens: 20, max_batch_size: Some(4), enable_cuda_graphs: false, hostname: "container-0.local", port: 8080, shard_uds_path: "/tmp/text-generation-server", master_addr: "localhost", master_port: 29500, huggingface_hub_cache: Some("/opt/ml/model"), weights_cache_override: None, disable_custom_kernels: false, cuda_memory_fraction: 1.0, rope_scaling: None, rope_factor: None, json_output: false, otlp_endpoint: None, cors_allow_origin: [], watermark_gamma: None, watermark_delta: None, ngrok: false, ngrok_authtoken: None, ngrok_edge: None, tokenizer_config_path: None, disable_grammar_support: false, env: false }

Copy
#033[2m2024-03-28T01:20:37.609995Z#033[0m #033[32m INFO#033[0m #033[2mtext_generation_launcher#033[0m#033[2m:#033[0m Args { model_id: "meta-llama/Llama-2-7b-chat-hf", revision: None, validation_workers: 2, sharded: None, num_shard: None, quantize: None, speculate: None, dtype: None, trust_remote_code: false, max_concurrent_requests: 128, max_best_of: 2, max_stop_sequences: 4, max_top_n_tokens: 5, max_input_length: 2048, max_total_tokens: 4096, waiting_served_ratio: 1.2, max_batch_prefill_tokens: 4096, max_batch_total_tokens: None, max_waiting_tokens: 20, max_batch_size: Some(4), enable_cuda_graphs: false, hostname: "container-0.local", port: 8080, shard_uds_path: "/tmp/text-generation-server", master_addr: "localhost", master_port: 29500, huggingface_hub_cache: Some("/opt/ml/model"), weights_cache_override: None, disable_custom_kernels: false, cuda_memory_fraction: 1.0, rope_scaling: None, rope_factor: None, json_output: false, otlp_endpoint: None, cors_allow_origin: [], watermark_gamma: None, watermark_delta: None, ngrok: false, ngrok_authtoken: None, ngrok_edge: None, tokenizer_config_path: None, disable_grammar_support: false, env: false }
2024-03-28T01:20:37.918Z
#033[2m2024-03-28T01:20:37.610075Z#033[0m #033[32m INFO#033[0m #033[1mdownload#033[0m: #033[2mtext_generation_launcher#033[0m#033[2m:#033[0m Starting download process.

Copy
#033[2m2024-03-28T01:20:37.610075Z#033[0m #033[32m INFO#033[0m #033[1mdownload#033[0m: #033[2mtext_generation_launcher#033[0m#033[2m:#033[0m Starting download process.
2024-03-28T01:20:39.421Z
#033[2m2024-03-28T01:20:37.701299Z#033[0m #033[33m WARN#033[0m #033[2mtext_generation_launcher#033[0m#033[2m:#033[0m 'extension' argument is not supported and will be ignored.

Copy
#033[2m2024-03-28T01:20:37.701299Z#033[0m #033[33m WARN#033[0m #033[2mtext_generation_launcher#033[0m#033[2m:#033[0m 'extension' argument is not supported and will be ignored.
2024-03-28T01:20:39.421Z
#033[2m2024-03-28T01:20:39.412447Z#033[0m #033[31mERROR#033[0m #033[1mdownload#033[0m: #033[2mtext_generation_launcher#033[0m#033[2m:#033[0m Download encountered an error:

Copy
#033[2m2024-03-28T01:20:39.412447Z#033[0m #033[31mERROR#033[0m #033[1mdownload#033[0m: #033[2mtext_generation_launcher#033[0m#033[2m:#033[0m Download encountered an error:
2024-03-28T01:20:39.421Z
/usr/local/lib/python3.10/dist-packages/transformers/utils/hub.py:123: FutureWarning: Using `TRANSFORMERS_CACHE` is deprecated and will be removed in v5 of Transformers. Use `HF_HOME` instead.

Copy
/usr/local/lib/python3.10/dist-packages/transformers/utils/hub.py:123: FutureWarning: Using `TRANSFORMERS_CACHE` is deprecated and will be removed in v5 of Transformers. Use `HF_HOME` instead.
2024-03-28T01:20:39.421Z
Error: DownloadError
  warnings.warn(

Copy
Error: DownloadError warnings.warn(
2024-03-28T01:20:39.421Z
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_errors.py", line 304, in hf_raise_for_status
    response.raise_for_status()
  File "/usr/local/lib/python3.10/dist-packages/requests/models.py", line 1021, in raise_for_status
    raise HTTPError(http_error_msg, response=self)

Copy
Traceback (most recent call last): File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_errors.py", line 304, in hf_raise_for_status response.raise_for_status() File "/usr/local/lib/python3.10/dist-packages/requests/models.py", line 1021, in raise_for_status raise HTTPError(http_error_msg, response=self)
2024-03-28T01:20:39.421Z
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://huggingface.co/meta-llama/Llama-2-7b-chat-hf/resolve/main/config.json

Copy
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://huggingface.co/meta-llama/Llama-2-7b-chat-hf/resolve/main/config.json
2024-03-28T01:20:39.421Z
The above exception was the direct cause of the following exception:

Copy
The above exception was the direct cause of the following exception:
2024-03-28T01:20:39.421Z
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/transformers/utils/hub.py", line 389, in cached_file
    resolved_file = hf_hub_download(
  File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_validators.py", line 118, in _inner_fn
    return fn(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py", line 1403, in hf_hub_download
    raise head_call_error
  File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py", line 1261, in hf_hub_download
    metadata = get_hf_file_metadata(
  File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_validators.py", line 118, in _inner_fn
    return fn(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py", line 1667, in get_hf_file_metadata
    r = _request_wrapper(
  File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py", line 385, in _request_wrapper
    response = _request_wrapper(
  File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py", line 409, in _request_wrapper
    hf_raise_for_status(response)
  File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_errors.py", line 321, in hf_raise_for_status
    raise GatedRepoError(message, response) from e

Copy
Traceback (most recent call last): File "/usr/local/lib/python3.10/dist-packages/transformers/utils/hub.py", line 389, in cached_file resolved_file = hf_hub_download( File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_validators.py", line 118, in _inner_fn return fn(*args, **kwargs) File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py", line 1403, in hf_hub_download raise head_call_error File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py", line 1261, in hf_hub_download metadata = get_hf_file_metadata( File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_validators.py", line 118, in _inner_fn return fn(*args, **kwargs) File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py", line 1667, in get_hf_file_metadata r = _request_wrapper( File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py", line 385, in _request_wrapper response = _request_wrapper( File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py", line 409, in _request_wrapper hf_raise_for_status(response) File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_errors.py", line 321, in hf_raise_for_status raise GatedRepoError(message, response) from e
2024-03-28T01:20:39.421Z
huggingface_hub.utils._errors.GatedRepoError: 401 Client Error. (Request ID: Root=1-6604c5e7-1e34c6bd142762892a5ebdef;ae197c89-158e-47ec-a1b0-b19f8048c2a0)

Copy
huggingface_hub.utils._errors.GatedRepoError: 401 Client Error. (Request ID: Root=1-6604c5e7-1e34c6bd142762892a5ebdef;ae197c89-158e-47ec-a1b0-b19f8048c2a0)
2024-03-28T01:20:39.421Z
Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-2-7b-chat-hf/resolve/main/config.json.

Copy
Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-2-7b-chat-hf/resolve/main/config.json.
2024-03-28T01:20:39.421Z
Repo model meta-llama/Llama-2-7b-chat-hf is gated. You must be authenticated to access it.

Copy
Repo model meta-llama/Llama-2-7b-chat-hf is gated. You must be authenticated to access it.
2024-03-28T01:20:39.421Z
The above exception was the direct cause of the following exception:

Copy
The above exception was the direct cause of the following exception:
2024-03-28T01:20:39.421Z
Traceback (most recent call last):
  File "/usr/local/bin/text-generation-server", line 8, in <module>
    sys.exit(app())
  File "/usr/local/lib/python3.10/dist-packages/text_generation_server/cli.py", line 101, in download_weights
    fetch_model(model_id, revision)
  File "/usr/local/lib/python3.10/dist-packages/text_generation_server/model.py", line 87, in fetch_model
    config = AutoConfig.from_pretrained(model_id, revision=revision)
  File "/usr/local/lib/python3.10/dist-packages/transformers/models/auto/configuration_auto.py", line 1082, in from_pretrained
    config_dict, unused_kwargs = PretrainedConfig.get_config_dict(pretrained_model_name_or_path, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/transformers/configuration_utils.py", line 644, in get_config_dict
    config_dict, kwargs = cls._get_config_dict(pretrained_model_name_or_path, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/transformers/configuration_utils.py", line 699, in _get_config_dict
    resolved_config_file = cached_file(
  File "/usr/local/lib/python3.10/dist-packages/transformers/utils/hub.py", line 404, in cached_file
    raise EnvironmentError(

Copy
Traceback (most recent call last): File "/usr/local/bin/text-generation-server", line 8, in <module> sys.exit(app()) File "/usr/local/lib/python3.10/dist-packages/text_generation_server/cli.py", line 101, in download_weights fetch_model(model_id, revision) File "/usr/local/lib/python3.10/dist-packages/text_generation_server/model.py", line 87, in fetch_model config = AutoConfig.from_pretrained(model_id, revision=revision) File "/usr/local/lib/python3.10/dist-packages/transformers/models/auto/configuration_auto.py", line 1082, in from_pretrained config_dict, unused_kwargs = PretrainedConfig.get_config_dict(pretrained_model_name_or_path, **kwargs) File "/usr/local/lib/python3.10/dist-packages/transformers/configuration_utils.py", line 644, in get_config_dict config_dict, kwargs = cls._get_config_dict(pretrained_model_name_or_path, **kwargs) File "/usr/local/lib/python3.10/dist-packages/transformers/configuration_utils.py", line 699, in _get_config_dict resolved_config_file = cached_file( File "/usr/local/lib/python3.10/dist-packages/transformers/utils/hub.py", line 404, in cached_file raise EnvironmentError(
2024-03-28T01:20:39.421Z
OSError: You are trying to access a gated repo.

Copy
OSError: You are trying to access a gated repo.
2024-03-28T01:20:40.423Z
Make sure to request access at https://huggingface.co/meta-llama/Llama-2-7b-chat-hf and pass a token having permission to this repo either by logging in with `huggingface-cli login` or by passing `token=<your_token>`.

Copy
Make sure to request access at https://huggingface.co/meta-llama/Llama-2-7b-chat-hf and pass a token having permission to this repo either by logging in with `huggingface-cli login` or by passing `token=<your_token>`.
2024-03-28T01:20:40.423Z
#033[2m2024-03-28T01:20:40.274297Z#033[0m #033[32m INFO#033[0m #033[2mtext_generation_launcher#033[0m#033[2m:#033[0m Args { model_id: "meta-llama/Llama-2-7b-chat-hf", revision: None, validation_workers: 2, sharded: None, num_shard: None, quantize: None, speculate: None, dtype: None, trust_remote_code: false, max_concurrent_requests: 128, max_best_of: 2, max_stop_sequences: 4, max_top_n_tokens: 5, max_input_length: 2048, max_total_tokens: 4096, waiting_served_ratio: 1.2, max_batch_prefill_tokens: 4096, max_batch_total_tokens: None, max_waiting_tokens: 20, max_batch_size: Some(4), enable_cuda_graphs: false, hostname: "container-0.local", port: 8080, shard_uds_path: "/tmp/text-generation-server", master_addr: "localhost", master_port: 29500, huggingface_hub_cache: Some("/opt/ml/model"), weights_cache_override: None, disable_custom_kernels: false, cuda_memory_fraction: 1.0, rope_scaling: None, rope_factor: None, json_output: false, otlp_endpoint: None, cors_allow_origin: [], watermark_gamma: None, watermark_delta: None, ngrok: false, ngrok_authtoken: None, ngrok_edge: None, tokenizer_config_path: None, disable_grammar_support: false, env: false }

Copy
#033[2m2024-03-28T01:20:40.274297Z#033[0m #033[32m INFO#033[0m #033[2mtext_generation_launcher#033[0m#033[2m:#033[0m Args { model_id: "meta-llama/Llama-2-7b-chat-hf", revision: None, validation_workers: 2, sharded: None, num_shard: None, quantize: None, speculate: None, dtype: None, trust_remote_code: false, max_concurrent_requests: 128, max_best_of: 2, max_stop_sequences: 4, max_top_n_tokens: 5, max_input_length: 2048, max_total_tokens: 4096, waiting_served_ratio: 1.2, max_batch_prefill_tokens: 4096, max_batch_total_tokens: None, max_waiting_tokens: 20, max_batch_size: Some(4), enable_cuda_graphs: false, hostname: "container-0.local", port: 8080, shard_uds_path: "/tmp/text-generation-server", master_addr: "localhost", master_port: 29500, huggingface_hub_cache: Some("/opt/ml/model"), weights_cache_override: None, disable_custom_kernels: false, cuda_memory_fraction: 1.0, rope_scaling: None, rope_factor: None, json_output: false, otlp_endpoint: None, cors_allow_origin: [], watermark_gamma: None, watermark_delta: None, ngrok: false, ngrok_authtoken: None, ngrok_edge: None, tokenizer_config_path: None, disable_grammar_support: false, env: false }
2024-03-28T01:20:40.423Z
#033[2m2024-03-28T01:20:40.274380Z#033[0m #033[32m INFO#033[0m #033[1mdownload#033[0m: #033[2mtext_generation_launcher#033[0m#033[2m:#033[0m Starting download process.

Copy
#033[2m2024-03-28T01:20:40.274380Z#033[0m #033[32m INFO#033[0m #033[1mdownload#033[0m: #033[2mtext_generation_launcher#033[0m#033[2m:#033[0m Starting download process.
2024-03-28T01:20:42.178Z
#033[2m2024-03-28T01:20:40.366345Z#033[0m #033[33m WARN#033[0m #033[2mtext_generation_launcher#033[0m#033[2m:#033[0m 'extension' argument is not supported and will be ignored.

Copy
#033[2m2024-03-28T01:20:40.366345Z#033[0m #033[33m WARN#033[0m #033[2mtext_generation_launcher#033[0m#033[2m:#033[0m 'extension' argument is not supported and will be ignored.
2024-03-28T01:20:42.178Z
#033[2m2024-03-28T01:20:42.076773Z#033[0m #033[31mERROR#033[0m #033[1mdownload#033[0m: #033[2mtext_generation_launcher#033[0m#033[2m:#033[0m Download encountered an error:

Copy
#033[2m2024-03-28T01:20:42.076773Z#033[0m #033[31mERROR#033[0m #033[1mdownload#033[0m: #033[2mtext_generation_launcher#033[0m#033[2m:#033[0m Download encountered an error:
2024-03-28T01:20:42.178Z
/usr/local/lib/python3.10/dist-packages/transformers/utils/hub.py:123: FutureWarning: Using `TRANSFORMERS_CACHE` is deprecated and will be removed in v5 of Transformers. Use `HF_HOME` instead.
  warnings.warn(

Copy
/usr/local/lib/python3.10/dist-packages/transformers/utils/hub.py:123: FutureWarning: Using `TRANSFORMERS_CACHE` is deprecated and will be removed in v5 of Transformers. Use `HF_HOME` instead. warnings.warn(
2024-03-28T01:20:42.178Z
Traceback (most recent call last):

Copy
Traceback (most recent call last):
2024-03-28T01:20:42.178Z
Error: DownloadError
  File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_errors.py", line 304, in hf_raise_for_status
    response.raise_for_status()
  File "/usr/local/lib/python3.10/dist-packages/requests/models.py", line 1021, in raise_for_status
    raise HTTPError(http_error_msg, response=self)

Copy
Error: DownloadError File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_errors.py", line 304, in hf_raise_for_status response.raise_for_status() File "/usr/local/lib/python3.10/dist-packages/requests/models.py", line 1021, in raise_for_status raise HTTPError(http_error_msg, response=self)
2024-03-28T01:20:42.178Z
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://huggingface.co/meta-llama/Llama-2-7b-chat-hf/resolve/main/config.json

Copy
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://huggingface.co/meta-llama/Llama-2-7b-chat-hf/resolve/main/config.json
2024-03-28T01:20:42.178Z
The above exception was the direct cause of the following exception:

Copy
The above exception was the direct cause of the following exception:
2024-03-28T01:20:42.178Z
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/transformers/utils/hub.py", line 389, in cached_file
    resolved_file = hf_hub_download(
  File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_validators.py", line 118, in _inner_fn
    return fn(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py", line 1403, in hf_hub_download
    raise head_call_error
  File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py", line 1261, in hf_hub_download
    metadata = get_hf_file_metadata(
  File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_validators.py", line 118, in _inner_fn
    return fn(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py", line 1667, in get_hf_file_metadata
    r = _request_wrapper(
  File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py", line 385, in _request_wrapper
    response = _request_wrapper(
  File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py", line 409, in _request_wrapper
    hf_raise_for_status(response)
  File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_errors.py", line 321, in hf_raise_for_status
    raise GatedRepoError(message, response) from e

Copy
Traceback (most recent call last): File "/usr/local/lib/python3.10/dist-packages/transformers/utils/hub.py", line 389, in cached_file resolved_file = hf_hub_download( File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_validators.py", line 118, in _inner_fn return fn(*args, **kwargs) File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py", line 1403, in hf_hub_download raise head_call_error File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py", line 1261, in hf_hub_download metadata = get_hf_file_metadata( File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_validators.py", line 118, in _inner_fn return fn(*args, **kwargs) File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py", line 1667, in get_hf_file_metadata r = _request_wrapper( File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py", line 385, in _request_wrapper response = _request_wrapper( File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py", line 409, in _request_wrapper hf_raise_for_status(response) File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_errors.py", line 321, in hf_raise_for_status raise GatedRepoError(message, response) from e
2024-03-28T01:20:42.178Z
huggingface_hub.utils._errors.GatedRepoError: 401 Client Error. (Request ID: Root=1-6604c5e9-3fc1188c687dda652e87a12c;eacfdaa2-8103-4044-b56a-9b8883e83b5e)

Copy
huggingface_hub.utils._errors.GatedRepoError: 401 Client Error. (Request ID: Root=1-6604c5e9-3fc1188c687dda652e87a12c;eacfdaa2-8103-4044-b56a-9b8883e83b5e)
2024-03-28T01:20:42.178Z
Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-2-7b-chat-hf/resolve/main/config.json.

Copy
Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-2-7b-chat-hf/resolve/main/config.json.
2024-03-28T01:20:42.178Z
Repo model meta-llama/Llama-2-7b-chat-hf is gated. You must be authenticated to access it.

Copy
Repo model meta-llama/Llama-2-7b-chat-hf is gated. You must be authenticated to access it.
2024-03-28T01:20:42.178Z
The above exception was the direct cause of the following exception:

Copy
The above exception was the direct cause of the following exception:
2024-03-28T01:20:42.178Z
Traceback (most recent call last):
  File "/usr/local/bin/text-generation-server", line 8, in <module>
    sys.exit(app())
  File "/usr/local/lib/python3.10/dist-packages/text_generation_server/cli.py", line 101, in download_weights
    fetch_model(model_id, revision)
  File "/usr/local/lib/python3.10/dist-packages/text_generation_server/model.py", line 87, in fetch_model
    config = AutoConfig.from_pretrained(model_id, revision=revision)
  File "/usr/local/lib/python3.10/dist-packages/transformers/models/auto/configuration_auto.py", line 1082, in from_pretrained
    config_dict, unused_kwargs = PretrainedConfig.get_config_dict(pretrained_model_name_or_path, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/transformers/configuration_utils.py", line 644, in get_config_dict
    config_dict, kwargs = cls._get_config_dict(pretrained_model_name_or_path, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/transformers/configuration_utils.py", line 699, in _get_config_dict
    resolved_config_file = cached_file(
  File "/usr/local/lib/python3.10/dist-packages/transformers/utils/hub.py", line 404, in cached_file
    raise EnvironmentError(

Motivation
To better use HF NeuronX DLC for commercial usage.

@Neo9061 did you set the HF_HUB_OFFLINE variable to True as I suggested in the first place ?

@dacorvo No I didn't recall it. Let me try it now.

@dacorvo with HF_HUB_OFFLINE enabled, we have another error. Please see below.
2024-03-28T14:46:26.258Z
#033[2m2024-03-28T14:46:25.480720Z#033[0m #033[32m INFO#033[0m #033[2mtext_generation_launcher#033[0m#033[2m:#033[0m Args { model_id: "meta-llama/Llama-2-7b-chat-hf", revision: None, validation_workers: 2, sharded: None, num_shard: None, quantize: None, speculate: None, dtype: None, trust_remote_code: false, max_concurrent_requests: 128, max_best_of: 2, max_stop_sequences: 4, max_top_n_tokens: 5, max_input_length: 2048, max_total_tokens: 4096, waiting_served_ratio: 1.2, max_batch_prefill_tokens: 4096, max_batch_total_tokens: None, max_waiting_tokens: 20, max_batch_size: Some(4), enable_cuda_graphs: false, hostname: "container-0.local", port: 8080, shard_uds_path: "/tmp/text-generation-server", master_addr: "localhost", master_port: 29500, huggingface_hub_cache: Some("/opt/ml/model"), weights_cache_override: None, disable_custom_kernels: false, cuda_memory_fraction: 1.0, rope_scaling: None, rope_factor: None, json_output: false, otlp_endpoint: None, cors_allow_origin: [], watermark_gamma: None, watermark_delta: None, ngrok: false, ngrok_authtoken: None, ngrok_edge: None, tokenizer_config_path: None, disable_grammar_support: false, env: false }

Copy
#033[2m2024-03-28T14:46:25.480720Z#033[0m #033[32m INFO#033[0m #033[2mtext_generation_launcher#033[0m#033[2m:#033[0m Args { model_id: "meta-llama/Llama-2-7b-chat-hf", revision: None, validation_workers: 2, sharded: None, num_shard: None, quantize: None, speculate: None, dtype: None, trust_remote_code: false, max_concurrent_requests: 128, max_best_of: 2, max_stop_sequences: 4, max_top_n_tokens: 5, max_input_length: 2048, max_total_tokens: 4096, waiting_served_ratio: 1.2, max_batch_prefill_tokens: 4096, max_batch_total_tokens: None, max_waiting_tokens: 20, max_batch_size: Some(4), enable_cuda_graphs: false, hostname: "container-0.local", port: 8080, shard_uds_path: "/tmp/text-generation-server", master_addr: "localhost", master_port: 29500, huggingface_hub_cache: Some("/opt/ml/model"), weights_cache_override: None, disable_custom_kernels: false, cuda_memory_fraction: 1.0, rope_scaling: None, rope_factor: None, json_output: false, otlp_endpoint: None, cors_allow_origin: [], watermark_gamma: None, watermark_delta: None, ngrok: false, ngrok_authtoken: None, ngrok_edge: None, tokenizer_config_path: None, disable_grammar_support: false, env: false }
2024-03-28T14:46:26.258Z
#033[2m2024-03-28T14:46:25.480802Z#033[0m #033[32m INFO#033[0m #033[1mdownload#033[0m: #033[2mtext_generation_launcher#033[0m#033[2m:#033[0m Starting download process.

Copy
#033[2m2024-03-28T14:46:25.480802Z#033[0m #033[32m INFO#033[0m #033[1mdownload#033[0m: #033[2mtext_generation_launcher#033[0m#033[2m:#033[0m Starting download process.
2024-03-28T14:46:27.255Z
#033[2m2024-03-28T14:46:25.572628Z#033[0m #033[33m WARN#033[0m #033[2mtext_generation_launcher#033[0m#033[2m:#033[0m 'extension' argument is not supported and will be ignored.

Copy
#033[2m2024-03-28T14:46:25.572628Z#033[0m #033[33m WARN#033[0m #033[2mtext_generation_launcher#033[0m#033[2m:#033[0m 'extension' argument is not supported and will be ignored.
2024-03-28T14:46:27.255Z
Error: DownloadError

Copy
Error: DownloadError
2024-03-28T14:46:27.255Z
#033[2m2024-03-28T14:46:27.183593Z#033[0m #033[31mERROR#033[0m #033[1mdownload#033[0m: #033[2mtext_generation_launcher#033[0m#033[2m:#033[0m Download encountered an error:

Copy
#033[2m2024-03-28T14:46:27.183593Z#033[0m #033[31mERROR#033[0m #033[1mdownload#033[0m: #033[2mtext_generation_launcher#033[0m#033[2m:#033[0m Download encountered an error:
2024-03-28T14:46:27.255Z
/usr/local/lib/python3.10/dist-packages/transformers/utils/hub.py:123: FutureWarning: Using `TRANSFORMERS_CACHE` is deprecated and will be removed in v5 of Transformers. Use `HF_HOME` instead.
  warnings.warn(

Copy
/usr/local/lib/python3.10/dist-packages/transformers/utils/hub.py:123: FutureWarning: Using `TRANSFORMERS_CACHE` is deprecated and will be removed in v5 of Transformers. Use `HF_HOME` instead. warnings.warn(
2024-03-28T14:46:27.255Z
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py", line 1261, in hf_hub_download
    metadata = get_hf_file_metadata(
  File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_validators.py", line 118, in _inner_fn
    return fn(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py", line 1667, in get_hf_file_metadata
    r = _request_wrapper(
  File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py", line 385, in _request_wrapper
    response = _request_wrapper(
  File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py", line 408, in _request_wrapper
    response = get_session().request(method=method, url=url, **params)
  File "/usr/local/lib/python3.10/dist-packages/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.10/dist-packages/requests/sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_http.py", line 78, in send
    raise OfflineModeIsEnabled(

Copy
Traceback (most recent call last): File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py", line 1261, in hf_hub_download metadata = get_hf_file_metadata( File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_validators.py", line 118, in _inner_fn return fn(*args, **kwargs) File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py", line 1667, in get_hf_file_metadata r = _request_wrapper( File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py", line 385, in _request_wrapper response = _request_wrapper( File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py", line 408, in _request_wrapper response = get_session().request(method=method, url=url, **params) File "/usr/local/lib/python3.10/dist-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) File "/usr/local/lib/python3.10/dist-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_http.py", line 78, in send raise OfflineModeIsEnabled(
2024-03-28T14:46:27.255Z
huggingface_hub.utils._http.OfflineModeIsEnabled: Cannot reach https://huggingface.co/meta-llama/Llama-2-7b-chat-hf/resolve/main/config.json: offline mode is enabled. To disable it, please unset the `HF_HUB_OFFLINE` environment variable.

Copy
huggingface_hub.utils._http.OfflineModeIsEnabled: Cannot reach https://huggingface.co/meta-llama/Llama-2-7b-chat-hf/resolve/main/config.json: offline mode is enabled. To disable it, please unset the `HF_HUB_OFFLINE` environment variable.
2024-03-28T14:46:27.255Z
The above exception was the direct cause of the following exception:

Copy
The above exception was the direct cause of the following exception:
2024-03-28T14:46:27.255Z
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/transformers/utils/hub.py", line 389, in cached_file
    resolved_file = hf_hub_download(
  File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_validators.py", line 118, in _inner_fn
    return fn(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py", line 1406, in hf_hub_download
    raise LocalEntryNotFoundError(

Copy
Traceback (most recent call last): File "/usr/local/lib/python3.10/dist-packages/transformers/utils/hub.py", line 389, in cached_file resolved_file = hf_hub_download( File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_validators.py", line 118, in _inner_fn return fn(*args, **kwargs) File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py", line 1406, in hf_hub_download raise LocalEntryNotFoundError(
2024-03-28T14:46:27.255Z
huggingface_hub.utils._errors.LocalEntryNotFoundError: An error happened while trying to locate the file on the Hub and we cannot find the requested files in the local cache. Please check your connection and try again or make sure your Internet connection is on.

Copy
huggingface_hub.utils._errors.LocalEntryNotFoundError: An error happened while trying to locate the file on the Hub and we cannot find the requested files in the local cache. Please check your connection and try again or make sure your Internet connection is on.
2024-03-28T14:46:27.255Z
The above exception was the direct cause of the following exception:

Copy
The above exception was the direct cause of the following exception:
2024-03-28T14:46:27.255Z
Traceback (most recent call last):
  File "/usr/local/bin/text-generation-server", line 8, in <module>
    sys.exit(app())
  File "/usr/local/lib/python3.10/dist-packages/text_generation_server/cli.py", line 101, in download_weights
    fetch_model(model_id, revision)
  File "/usr/local/lib/python3.10/dist-packages/text_generation_server/model.py", line 87, in fetch_model
    config = AutoConfig.from_pretrained(model_id, revision=revision)
  File "/usr/local/lib/python3.10/dist-packages/transformers/models/auto/configuration_auto.py", line 1082, in from_pretrained
    config_dict, unused_kwargs = PretrainedConfig.get_config_dict(pretrained_model_name_or_path, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/transformers/configuration_utils.py", line 644, in get_config_dict
    config_dict, kwargs = cls._get_config_dict(pretrained_model_name_or_path, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/transformers/configuration_utils.py", line 699, in _get_config_dict
    resolved_config_file = cached_file(
  File "/usr/local/lib/python3.10/dist-packages/transformers/utils/hub.py", line 429, in cached_file
    raise EnvironmentError(

Copy
Traceback (most recent call last): File "/usr/local/bin/text-generation-server", line 8, in <module> sys.exit(app()) File "/usr/local/lib/python3.10/dist-packages/text_generation_server/cli.py", line 101, in download_weights fetch_model(model_id, revision) File "/usr/local/lib/python3.10/dist-packages/text_generation_server/model.py", line 87, in fetch_model config = AutoConfig.from_pretrained(model_id, revision=revision) File "/usr/local/lib/python3.10/dist-packages/transformers/models/auto/configuration_auto.py", line 1082, in from_pretrained config_dict, unused_kwargs = PretrainedConfig.get_config_dict(pretrained_model_name_or_path, **kwargs) File "/usr/local/lib/python3.10/dist-packages/transformers/configuration_utils.py", line 644, in get_config_dict config_dict, kwargs = cls._get_config_dict(pretrained_model_name_or_path, **kwargs) File "/usr/local/lib/python3.10/dist-packages/transformers/configuration_utils.py", line 699, in _get_config_dict resolved_config_file = cached_file( File "/usr/local/lib/python3.10/dist-packages/transformers/utils/hub.py", line 429, in cached_file raise EnvironmentError(
2024-03-28T14:46:27.255Z
OSError: We couldn't connect to 'https://huggingface.co' to load this file, couldn't find it in the cached files and it looks like meta-llama/Llama-2-7b-chat-hf is not the path to a directory containing a file named config.json.

A side question @dacorvo, imagine multiple caches reading is working. When I prepare the neuron caches, I only want neuron caches for that particular models and potentially for the configurations that I selected. Is there an easy way to prepare such selected neuron caches? rather than downloading from aws-neuron/optimum-neuron cache? As aws-neuron/optimum-neuron cache has over 60GB of caches, and I cannot include all of them into a model's model input data.

@Neo9061 I have difficulties following what you are actually trying to do: if you want to have access to neuron models offline, then just export them and pass the path to the exported model directory instead of the model_id.
I don't understand why you would want to use the cache, which is a convoluted way to obtain the exact same thing. What are you trying to achieve ?

Let me clarify a bit more: based on this blogpost, where you essentially can configure a same model with different ENVs of sequence length, batch size, and tensor parallel degree, we want to make it work under network isolation.
Thus, when a user specify a config via ENVs, the DLC will find the corresponded pre-compiled neuron caches and start the an endpoint. To achieve that, we need store multiple neuron caches in local dir within DLC, along with model weights. And my follow up question in the above thread is how to filter or trim the neuron caches to a set of pre-selected configs -- as including all of configs for neuron caches are too big (60GB in aws-neuron/optimum-neuron)

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread. Thank you!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread. Thank you!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread. Thank you!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread. Thank you!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread. Thank you!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread. Thank you!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread. Thank you!


Fast closest_point operator for Quadrilateral and/or Polygon

Does anyone know of a fast algorithm for computing the closest point to a (possibly non-planar and possibly concave) quadrilateral or polygon? I have an application that currently computes the algebraic center of a quadrilateral and then uses that to divide the quadrilateral into 4 triangles. The closest point is then computed for each triangle and we take the one with the minimum squared distance. But this algorithm is expensive (currently accounts for about 15% of the run time).

Based on an offline discussion w/ @adayton1, the quads/polygons are embedded in 3D and the query points are also 3D.


Overriding resolver methods in extending classes.

Currently there is no way to override methods of extended BaseResolver class.
Here's an example:
import { Arg, Int, Query, Resolver } from 'type-graphql'
import { ObjectID, Repository }      from 'typeorm'
import { InjectRepository }          from 'typeorm-typedi-extensions'
import * as pluralize  from 'pluralize'

export default function createBaseResolver<T extends object> (suffix: string, objectTypeCls: new() => T)
{

	@Resolver({ isAbstract: true })
	abstract class BaseResolver
	{
		@InjectRepository(objectTypeCls)
		protected readonly repository: Repository<T>

		@Query(type => [objectTypeCls], { name: `${pluralize(suffix)}` })
		async getAll (): Promise<T[]>
		{
			return this.repository.find()
		}

		@Query(type => objectTypeCls, { nullable: true, name: `${suffix}` })
		async getOne (@Arg('id', type => Int) id: string | number | ObjectID): Promise<T>
		{
			return this.repository.findOne(id, { cache: true })
		}
	}

	return BaseResolver
}

Two example methods that don't do what I think should be expected. (Issue explained in the comment above the method declarations).
import {
	Resolver,
	Query,
	Arg,
	Mutation,
}                         from 'type-graphql'
import createBaseResolver from '../../basics/resource.resolver'
import { Language }       from './language'
import { LanguageInput }  from './language.input'

const LanguageBaseResolver = createBaseResolver('language', Language)

@Resolver(Language)
export class LanguageResolver extends LanguageBaseResolver
{

// Method override attempt #1

// This method gets overriden by one generated by BaseResolver,
// and still procudes [language(id: Int!): Language]
// instead of [language(identifier: String!): Language]
	@Query(returns => Language)
	language (@Arg('identifier') identifier: string): Promise<Language>
	{
		return this.repository.findOne({ identifier })
	}

// Method override attempt #2

// Even though this method shares the name with the one in BaseResolver and technically overrides it,
// the query it produces is [getOne(identifier: String!): Language!]
// instead of [language(identifier: String!): Language]
	@Query(returns => Language)
	getOne (@Arg('identifier') identifier: string): Promise<Language>
	{
		return this.repository.findOne({ identifier })
	}

}

type-graphql: 0.12.0

Overriding method from generated base resolver class...

That's why I thought that introducing this feature isn't a good idea as it overcomplicates things 😞
It should be fixable as now I concat the inherited resolvers after building child resolvers metadata but detecting overriding in JS/TS... I'm not sure whether should I detect only schema name overriding or also the class method name should supress the base one 😕

Overriding method from generated base resolver class...

It might sound a bit funky, but once in a while, overriding one of many inherited methods comes in handy. :smile:
Wouldn't mixins + interface be a better approach than inheritance anyway?

What do you mean by mixins + interface? Can you show an example API? 😉

Closing for now via 6debc6e 🎉
@IgorSzymanski
If you have a proposal mixins API, please open a new feature when we could discuss that.

@IgorSzymanski hi, i use your example in my code. I can't understand why InjectRepository not working in constructor, do you know why?


Why does ag --hidden pattern not search .local dir under current dir?

I'm in a directory with a subdirectory called .local. If I do ag --hidden pattern the .local subdirectory is not searched. Why? How can I include it?

Works perfectly fine for me:
/tmp/foo$ mkdir .local
/tmp/foo$ echo hallo >.local/world.txt
/tmp/foo$ echo hallo >world.txt
/tmp/foo$ ag hallo
world.txt
1:hallo
/tmp/foo$ ag --hidden hallo
world.txt
1:hallo

.local/world.txt
1:hallo

.local must be in your .ignore file. Try --unrestricted.

Oh, it doesn't search files which are in .gitignore.


Set default branch to main

As noted in this email, Hyperledger Labs should, similar to Hyperledger Projects, move from master to main for the default branch. MinBFT also needs to do the switch.

@sergefdrv @nhoriguchi I'll contact Ry for updating the setting if you are OK to do it now.

We don't have any open pull requests, so it should be fine.

I've sent a request. The setting will be updated soon.

Done.


quoting the scalar

Not quoting the scalar "%*%" starting with the "%" indicator character is deprecated since Symfony 3.1 and will throw a ParseException in 4.0.

ping @Moinax


Separate release build target, use dev for tests

This should speed up testing workflows significantly.

Turned out to be counter-productive, so off it goes.


📎 Implement eslint-plugin-react-refresh

Description
from https://github.com/biomejs/biome/discussions/3#discussioncomment-9177559
react-refresh
react-refresh is an official package of react that provides bundler with a Fast Refresh feature that allows real-time updates of running React apps.
In addition to react-native, next.js, @vitejs/plugin-react and @parcel/runtime-react-refresh are often used behind the scenes in development environments.
It is a so-called hot reload, but it is characterized by the fact that only edited components can be updated minute by minute while maintaining the state.
For more detailed information, please refer to the documentation of each bundler.

React Native
next.js
Remix
@vitejs/plugin-react
Gatsby

eslint-plugin-react-refresh
eslint-plugin-react-refresh is a plugin also linked from @vitejs/plugin-react that defines rules to make react-refresh function properly.
Rule react-refresh/only-export-components is too large.
Already implemented

noReExportAll
noDefaultExport

What we really want is to prohibit anonymous default exports. (like import/no-anonymous-default-export)

I plan to implement it.

[ ] useComponentsOnlyModule

The following are not included in the current lint rules, but they are desired for future implementation.

improve useNamingConvention

Since PascalCase is used to identify React components, we would like something equivalent to react/jsx-pascal-case.
https://github.com/biomejs/biome/discussions/3#discussioncomment-9177700

noClassComponent

It seems that ClassComponents are not supported.

more info: https://github.com/biomejs/biome/discussions/2548

The main lint rule also outputs errors equivalent to noReExportAll. What would be an appropriate way to guide the enabling of this during migration?

For this extension, we want to apply it only to .jsx and .tsx files. Since it seems that this cannot be achieved by specifying language: "jsx", is it appropriate to implement this check within the run function?

There is a function that we want to share with other nursery rules (like noYodaExpression). Where would be the appropriate place to extract this function?

For this extension, we want to apply it only to .jsx and .tsx files

Note that while TypeScript is strict about requiring a .tsx extension for files with JSX syntax, such a requirement doesn't exist for JavaScript. A .js file with JSX syntax can be totally fine depending on one's bundler configuration.

The main lint rule also outputs errors equivalent to noReExportAll. What would be an appropriate way to guide the enabling of this during migration?

Unfortunately Biome migration tool doesn't support a source to be associated to several Biome rules. Thus, for now, it is not possible to do that.

For this extension, we want to apply it only to .jsx and .tsx files. Since it seems that this cannot be achieved by specifying language: "jsx", is it appropriate to implement this check within the run function?

Could you explain the reasoning behind this choice?
If it is really necessary, you can retrieve the file path (something like ctx.file_path()) and check the extension with the standard library.

There is a function that we want to share with other nursery rules (noYodaExpression). Where would be the appropriate place to extract this function?

If the function is quite specific, you can just make it public in one of the rule (pub(crate)) and import it in the other.
If the function is quite generic and seems useful in a general way, then you can place it in the biome_js_syntax crate.
We have a bunch of files ending with _ext where it could be placed.

In addition, since the Pascal case is used to determine whether a component is a component or not, we plan to implement functions related to the constraints around it.

biome_string_case::Case::identify should be helpful in that regard :)

Unfortunately Biome migration tool doesn't support a source to be associated to several Biome rules. Thus, for now, it is not possible to do that.

mmm...

Could you explain the reasoning behind this choice?
If it is really necessary, you can retrieve the file path (something like ctx.file_path()) and check the extension with the standard library.

In detecting React components, only the name is used, which can lead to false positives. The original lint rule was implemented to avoid this issue.

I have created a pull request and would like you to review it.
Codspeed has not passed, but how should I resolve this?

Unfortunately Biome migration tool doesn't support a source to be associated to several Biome rules. Thus, for now, it is not possible to do that.

Although, it could be fairly easy to change how we generate migrate and website pages to allow this.


drivers: flash: flash_mcux_flexspi: since adding support for W25Q512JV, FlexSPI with the W25Q128JV no longer works

Describe the bug

Hello,
Our custom board, using an IMX RT1040, with the W25Q128JV flash chip over FlexSPI as it's main flash, fails to boot after updating our Zephyr version to latest LTS 3.7.1 release (commit 9f824289b28d7aea2eee74f62787c385a5005453). After spending time to narrow it down to a specific commit that caused our issue, the conclusion we came to was that 29b2bf89f8401d35bf3dbb2fe17922944d52edfb broke the flash initialization. After removing that added switch case inside the flash_flexspi_nor_check_jedec function, our board is able to boot and operate exactly as it was prior, being on version 3.7 release. As far as I am aware, the command '0xEC' for Quad page programming, does not exist on the W25Q128JV whereas it does on the W25Q512JV.
To Reproduce

Use a W25Q128JV over FlexSPI using the nxp_imx_flexspi_nor driver
Attempt to boot on latest LTS 3.7.1

Expected behavior

This added switch case does not seem to apply to the W25Q128JV, instead of assuming that the vendor ID means this change will work for all W25Q* flash chips, more indepth checks are needed to validate what flash chip is currently in use before setting custom LUT tables.
Impact

Board does not boot. We have created a fork of LTS 3.7.1 with commit 29b2bf89f8401d35bf3dbb2fe17922944d52edfb reverted and will continue to use that until this is fixed.
Environment (please complete the following information):
OS: Windows
Toolchain: Zephyr SDK
Commit: 9f824289b28d7aea2eee74f62787c385a5005453
Additional context

I did read briefly on the original PR for adding support for the W25Q512JV that there is other correct ways to do this, is there a time frame on implementing such additions? The PR in question: https://github.com/zephyrproject-rtos/zephyr/pull/80261

https://github.com/zephyrproject-rtos/zephyr/pull/81102 should fix this on 4.0, but I think the better way to do this is to check all 3 bytes of the JEDEC ID, rather than the two most significant. I'll create a PR to do that. Long term we shouldn't be placing custom LUT tables in this driver at all, but that's a different conversation

Awesome, we moved to 4.0 and the issue isn't present anymore. Let me know if you need any testers for your future fix. Thanks again!

I'm actually going to reopen this issue, because I'll need something to link to with the backport PR to fix this issue on 3.7 (we require backport PRs to reference a specific issue)


EL1 requirement: Add HTTP headers to logs

In order to comply with EO M-21-31, we need to include HTTP headers in our logs where applicable.
Background

Presidential Memorandum M-21-31.
Notes from discussion with assurance team about M-21-31

Acceptance Criteria

[ ] GIVEN that we want to comply with EO M-21-31
WHEN I make a web request to an application
THEN I see HTTP headers in the logs

Security considerations
Including HTTP headers in request logs gives additional traceability for our system and could help to identify requests from malicious actors when necessary.
Implementation sketch

[ ] Determine if our logs already include HTTP headers when applicable
[ ] Make a test request using curl with a fake HTTP header to see if it is logged.

If unknown headers are stripped by the load balancer before getting to our Gorouters, that's a good thing, as it keeps them out of our security boundary

[ ] If necessary, update our configuration to include headers for HTTP requests
[ ] Document what headers or types of headers are logged/captured
[ ] Research and document how to handle logging of POST requests. Reference OWASP or other security standards

Cloudfront also filters headers — will also need to check impact on this.

Filtering headers so they never hit our infra/apps is going to be OK. We
just need to point to the documentation of what get's blocked, etc.
On Mon, Aug 1, 2022 at 12:55 PM James Hochadel @.***>
wrote:

Cloudfront also filters headers — will also need to check impact on this.
—
Reply to this email directly, view it on GitHub
https://github.com/cloud-gov/product/issues/2123#issuecomment-1201464390,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAJHWCTZUQLCEOXVKYFEKCDVW76QVANCNFSM54MTANRA
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

--
Peter Burkholder |  *
cloud.gov https://cloud.gov ISSO, compliance & security lead
please use @.** for cloud.gov matters
202-709-2028 <(202)%20209-2028> | @.**
@.***> *
| pronouns he-him https://www.mypronouns.org/he-him
Free/Busy Calendar
@.*>

By default security headers are added as described in this documentation: https://cloud.gov/docs/management/headers/.
We allow users to specify a limited number of headers to forward to their application when using the external_domain_broker: https://cloud.gov/docs/services/external-domain-service/#forward_headers-option

Information provided by gorouter access logs: https://github.com/cloudfoundry/gorouter#access-logs
It appears that no POST data is stored.

Should the drop_invalid_header field be enabled on the load balancers? https://aquasecurity.github.io/tfsec/v0.63.1/checks/aws/elb/drop-invalid-headers/

Update on headers: They do get passed through all the way to the app, but they are not logged. To log extra headers we will need to configure router.extra_headers_to_log. It takes an array as its argument, and we'll need to test if we can throw in a wildcard value to catch all headers.

Blocked pending discussion about logging extra headers

From @pburkholder: We should see if we can log these additional headers. These headers are passed to the app, but not logged. So if they are hijacked somehow, but we have no log of it, then we wouldn't be able to investigate the attack afterwards.
Decision: We should document what headers we log and then for any additional headers, state that logging them must be handled at the application level.
Action: Open an issue with gorouter to see if they can log all headers.

Opened an issue for the routing release here

We have created the necessary documentation so we are closing this issue.


rusoto::DispatchSignedRequest - Dead link in documentation

Hi,
when clicking on "Client" here (https://rusoto.github.io/rusoto/rusoto/trait.DispatchSignedRequest.html), I get a 404 (link to hyper documentation).

I think this is due to hyper using a directory to specify which version of the documentation to look at.  Our link goes to https://hyper.rs/hyper/hyper/client/struct.Client.html when it should go to https://hyper.rs/hyper/v0.9.12/hyper/client/struct.Client.html (assuming we're using 0.9.12).
I'm not sure how to fix this.  Will check IRC or another method to get help on the docs link.

Alright, thanks.

I've reached out for help on this.

@durka Can you point me in the right direction for possible fixes so I can present those in the hyper project?  Thanks.

I think hyper should write
#![doc(html_root_url = "https://hyper.rs/hyper/{{ current version}}/hyper")]

in their lib.rs, at least in the versions released on crates.io. Maybe there's an easier way, I'm not sure. cc @seanmonstar

🎉 this is fixed on Hyper's end.  We'll need to do a new build on master to pull in the recently published version of Hyper, 0.10.11, then the link should work when docs are published.  I tested Rusoto docs locally and it works.

Confirmed it's fixed: https://rusoto.github.io/rusoto/rusoto_core/request/trait.DispatchSignedRequest.html 's link for Client goes to https://hyper.rs/hyper/v0.10.11/hyper/client/struct.Client.html .


Git extractor

This PR updates the local GitExtractor to use the same schema.org properties and interface as other extractors. The GitExtractor works with a local folder (cloned repo) to extract metadata from the Git Index. It should also be called automatically from Project() when using gimie with an unsupported Git provider (gitea, codeberg, sourceforge, bitbucket, ...).
The PR also attempts to simplify how new sources are added and called from project.
The main unsolved challenges in this PR are:

how to best define a URI for a local project. Currently defined using the sha of the HEAD commit
how to define schema:creator. Currently, it is the author of the first commit.

Nice catch! This should be fixed now


[Bug]: Change of focus loses work in progress

Summary
If I view another app, then any work in progress in this app is lost
Steps to reproduce

Select image(s) to share
Share to app
Enter metadata for one or more images
Switch to another app (for example, to check the spelling of a name)
Cannot switch back to Commons app with the same state that it was left; whatever was typed is lost; images are no longer selected

Expected behaviour
The user should be able to return to the app and continue from where they left off.
Actual behaviour
Cannot.
Device name
Nord Plus One
Android version
Android 11
Commons app version
3.1.1~1c9267ca0
Device logs
No response
Screen-shots
No response
Would you like to work on the issue?
Prefer not

I totally agree, this is very frustrating!
State should be remembered (maybe in database?) and survive low memory kicks and ideally even reboots.
It will take a lot of effort to implement&test&maintain unfortunately, and we are all taking care of this app in our spare time. 😔


[xsimd] update to 12.0.0

Fixes https://github.com/microsoft/vcpkg/issues/35407

[x] Changes comply with the maintainer guide
[x] SHA512s are updated for each updated download
[ ] The "supports" clause reflects platforms that may be fixed by this new version
[ ] Any fixed CI baseline entries are removed from that file.
[ ] Any patches that are no longer applied are deleted from the port's directory.
[x] The version database is fixed by rerunning ./vcpkg x-add-version --all and committing the result.
[x] Only one version is added to each modified port's versions file.

Thanks for the update!


[J7F9] Pipeline reproduction

Softwares
SPM12, Update 7219
fmriprep, version 1.1.4
Matlab R2017b
Input data
derivatives (fMRIprep)
Additional context

Refactorisation is needed to conform with the narps_open/runner module.
Only hypotheses 6 and 8 provide unsufficient correlations (~= 0.88) with the original team results.

List of tasks
Please tick the boxes below once the corresponding task is finished. :+1:

[x] :ok_hand: A maintainer of the project approved the issue, by assigning a :checkered_flag:status: ready for dev label to it.
[x] :deciduous_tree: Create a branch on your fork to start the reproduction.
[x] :sunrise: Create a file team_{team_id}.py inside the narps_open/pipelines/ directory. You can use a file inside narps_open/pipelines/templates as a template if needed.
[x] :inbox_tray: Create a pull request as soon as you completed the previous task.
[x] :brain: Write the code for the pipeline, using Nipype and the file architecture described in docs/pipelines.md.
[x] :blue_book: Make sure your code is documented enough.
[x] :snake: Make sure your code is explicit and conforms with PEP8.
[x] :microscope: Create tests for your pipeline. You can use files in tests/pipelines/test_team_* as examples.
[x] :microscope: Make sure your code passes all the tests you created (see docs/testing.md).

PR https://github.com/Inria-Empenn/narps_open_pipelines/pull/80 merged to main : J7F9 refactored.
The code allows to get the following correlation results (hypothesis 1 to 9).
0.9131411735231
0.957296569265423
0.9131411735231
0.957296569265423
0.9843141125089053
0.9058252041092889
0.9843141125089053
0.9058252041092889
0.952892940943768

[!NOTE]
Subjects 005 and 043 produce errors during the subject level analysis (find crash files attached), hence the previous results are computed with 106 subjects.

crash-20240105-132640-root-model_design.a1-633d06b2-d0c4-4794-9c44-6a6c934f184c.txt
crash-20240105-132603-root-model_design.a0-ed511adc-d88f-4eb5-bd26-6e9d0e236546.txt
attached)

Given that all correlations are above 0.90, I think that we can consider this as a successful reproduction and therefore close this PR. @bclenet let me know if I missed something!