Job Errors on OpenStack Private Cloud (PAI v0.14.0)

Shaiic:

Error message of "1514 Illegal instruction (core dumped)" while running a "hello world" job with gpu: 0 using PAI v0.14.0:
**I‘m running a "Hello World" job(see below) with PAI v0.14.0, and I get an error message of "1514 Illegal instruction (core dumped)"
Docker image: openpai/tensorflow-py36-cpu
Command:
pip3 install contextlib2
apt update
apt install -y git
git clone https://github.com/tensorflow/models
cd models/research/slim
pip3 install contextlib2 Pillow
python download_and_convert_data.py --dataset_name=cifar10 --dataset_dir=/tmp/data
python train_image_classifier.py --dataset_name=cifar10 --dataset_dir=/tmp/data --max_number_of_steps=1000
**:

**
Process to reproduce:

Open "Platform for AI" GUI and access "submit job" menu

Fill in the following commands in "Command" textfield:
apt update
apt install -y git
git clone https://github.com/tensorflow/models
cd models/research/slim
pip3 uninstall tensorflow
pip3 install contextlib2 Pillow tensorflow==1.14
python download_and_convert_data.py --dataset_name=cifar10 --dataset_dir=/tmp/data
python train_image_classifier.py --dataset_name=cifar10 --dataset_dir=/tmp/data --max_number_of_steps=1000 --clone_on_cpu=True

Set "GPU count" to "0" in the "Resources per instance" filed

Toggle "Custom" in the "Docker image" field and enter "openpai/tensorflow-py36-cpu" in the text
box

Click "Submit" button, and after a while the error message is shown:

Exit Code:
132
Exit Type:
USER_FAILURE
Exit Reason:
Container killed by OS Signal: SIGILL
Exit Solutions:
Check container log and fix your program bug

Click "stderr“， you'll see the message below:
WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.
debconf: unable to initialize frontend: Dialog
debconf: (No usable dialog-like program is installed, so the dialog based frontend cannot be used. at /usr/share/perl5/Debconf/FrontEnd/Dialog.pm line 76, <> line 1.)
debconf: falling back to frontend: Readline
Cloning into 'models'...

WARNING: Skipping tensorflow as it is not installed.
WARNING: You are using pip version 19.2.2, however version 19.3.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
/pai/bootstrap/docker_bootstrap.sh: line 206:   750 Illegal instruction     (core dumped) python download_and_convert_data.py --dataset_name=cifar10 --dataset_dir=/tmp/data
**:

OpenPAI Environment:

OpenPAI version: v0.14.0
Cloud provider or hardware configuration: Yidian Cloud powered by i-stack Cloud OS (An OpenStack Private Cloud)
OS (e.g. from /etc/os-release):
NAME="Ubuntu"
VERSION="16.04.1 LTS (Xenial Xerus)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 16.04.1 LTS"
VERSION_ID="16.04"
HOME_URL="http://www.ubuntu.com/"
SUPPORT_URL="http://help.ubuntu.com/"
BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"
VERSION_CODENAME=xenial
UBUNTU_CODENAME=xenial
Kernel (e.g. uname -a):Linux liuc01-wushan 4.4.0-170-generic #199-Ubuntu SMP Thu Nov 14 01:45:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Hardware (e.g. core number, memory size, storage size, GPU type etc.): 16 cores cpu/128g memory/100g disk
Others:

Anything else we need to know:

Hi, according to your log
Checking out files: 100% (3194/3194), done.
WARNING: Skipping tensorflow as it is not installed.
WARNING: You are using pip version 19.2.2, however version 19.3.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
/pai/bootstrap/docker_bootstrap.sh: line 206: 750 Illegal instruction (core dumped) python download_and_convert_data.py --dataset_name=cifar10 --dataset_dir=/tmp/data

seems it does not execute the pip install part in commands:
pip3 uninstall tensorflow
pip3 install contextlib2 Pillow tensorflow==1.14

Your reproduction commands work for me,
apt update
apt install -y git
git clone https://github.com/tensorflow/models
cd models/research/slim
pip3 uninstall tensorflow
pip3 install contextlib2 Pillow tensorflow==1.14
python download_and_convert_data.py --dataset_name=cifar10 --dataset_dir=/tmp/data
python train_image_classifier.py --dataset_name=cifar10 --dataset_dir=/tmp/data --max_number_of_steps=1000 --clone_on_cpu=True

could you pls check pip install works fine in your environment, e.g. network?

Thx Abuttcs for reply! I login to the container and "pip install" command works fine. I add "pip install --upgrade pip"  and remoe "pip3 uninstall tensorflow" in the "Command" textfield, and the warning message has gone away but the same error still exists.
Given below is the current "Command":

apt update
apt install -y git
git clone https://github.com/tensorflow/models
cd models/research/slim
pip install --upgrade pip
pip3 install contextlib2 Pillow tensorflow==1.14
python download_and_convert_data.py --dataset_name=cifar10 --dataset_dir=/tmp/data
python train_image_classifier.py --dataset_name=cifar10 --dataset_dir=/tmp/data --max_number_of_steps=1000 --clone_on_cpu=True

Given below is the Stdout log:

B]
Fetched 3912 kB in 4s (963 kB/s)
(Reading database ...

Preparing to unpack .../git_1%3a2.17.1-1ubuntu0.5_amd64.deb ...
Unpacking git (1:2.17.1-1ubuntu0.5) over (1:2.17.1-1ubuntu0.4) ...
Setting up git (1:2.17.1-1ubuntu0.5) ...
Collecting pip
Downloading https://files.pythonhosted.org/packages/54/0c/d01aa759fdc501a58f431eb594a17495f15b88da142ce14b5845662c13f3/pip-20.0.2-py2.py3-none-any.whl (1.4MB)
Installing collected packages: pip
Found existing installation: pip 19.2.2
Uninstalling pip-19.2.2:
Successfully uninstalled pip-19.2.2
Successfully installed pip-20.0.2
Collecting contextlib2
Downloading contextlib2-0.6.0.post1-py2.py3-none-any.whl (9.8 kB)
Collecting Pillow
Downloading Pillow-7.0.0-cp36-cp36m-manylinux1_x86_64.whl (2.1 MB)
Collecting tensorflow==1.14
Downloading tensorflow-1.14.0-cp36-cp36m-manylinux1_x86_64.whl (109.2 MB)
Requirement already satisfied: google-pasta>=0.1.6 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.14) (0.1.7)
Requirement already satisfied: gast>=0.2.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.14) (0.2.2)
Collecting tensorflow-estimator<1.15.0rc0,>=1.14.0rc0
Downloading tensorflow_estimator-1.14.0-py2.py3-none-any.whl (488 kB)
Requirement already satisfied: six>=1.10.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.14) (1.12.0)
Requirement already satisfied: wrapt>=1.11.1 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.14) (1.11.2)
Requirement already satisfied: grpcio>=1.8.6 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.14) (1.22.0)
Requirement already satisfied: termcolor>=1.1.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.14) (1.1.0)
Collecting tensorboard<1.15.0,>=1.14.0
Downloading tensorboard-1.14.0-py3-none-any.whl (3.1 MB)
Requirement already satisfied: protobuf>=3.6.1 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.14) (3.9.1)
Requirement already satisfied: keras-preprocessing>=1.0.5 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.14) (1.1.0)
Requirement already satisfied: absl-py>=0.7.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.14) (0.7.1)
Requirement already satisfied: numpy<2.0,>=1.14.5 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.14) (1.17.0)
Requirement already satisfied: wheel>=0.26 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.14) (0.33.4)
Requirement already satisfied: keras-applications>=1.0.6 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.14) (1.0.8)
Requirement already satisfied: astor>=0.6.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.14) (0.8.0)
Requirement already satisfied: setuptools>=41.0.0 in /usr/local/lib/python3.6/dist-packages (from tensorboard<1.15.0,>=1.14.0->tensorflow==1.14) (41.1.0)
Requirement already satisfied: markdown>=2.6.8 in /usr/local/lib/python3.6/dist-packages (from tensorboard<1.15.0,>=1.14.0->tensorflow==1.14) (3.1.1)
Requirement already satisfied: werkzeug>=0.11.15 in /usr/local/lib/python3.6/dist-packages (from tensorboard<1.15.0,>=1.14.0->tensorflow==1.14) (0.15.5)
Requirement already satisfied: h5py in /usr/local/lib/python3.6/dist-packages (from keras-applications>=1.0.6->tensorflow==1.14) (2.9.0)
Installing collected packages: contextlib2, Pillow, tensorflow-estimator, tensorboard, tensorflow
Successfully installed Pillow-7.0.0 contextlib2-0.6.0.post1 tensorboard-1.14.0 tensorflow-1.14.0 tensorflow-estimator-1.14.0

it shows that the network works fine.
Given below is the Stderr message:

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.
WARNING: apt does not have a stable CLI interface. Use with caution in scripts.
debconf: unable to initialize frontend: Dialog
debconf: (No usable dialog-like program is installed, so the dialog based frontend cannot be used. at /usr/share/perl5/Debconf/FrontEnd/Dialog.pm line 76, <> line 1.)
debconf: falling back to frontend: Readline
Cloning into 'models'...

And could you also paste the logs by clicking "View Exit Diagnostics" in job detail page?

I use a 12GB memory but still the error occurs.
Given below are the logs of "View Exit Diagnostics":

[Exit Trigger Info]
ExitTriggerMessage: FailedTaskCount 1 has reached MinFailedTaskCount 1 in TaskRole [Task_role_1]
ExitTriggerTaskRole: Task_role_1
ExitTriggerTaskIndex: 0
UserExitCode: 132

[Exit Spec]
code: 132
phrase: CONTAINER_KILLED_BY_SIGILL
issuer: USER_CONTAINER
causer: USER_CONTAINER
type: USER_FAILURE
stage: RUNNING
behavior: NON_TRANSIENT
reaction: NEVER_RETRY
reason: 'Container killed by OS Signal: SIGILL'
repro:

User program executes an illegal, malformed, unknown, or privileged machine
instruction
solution:
Check container log and fix your program bug

[Exit Diagnostics]
[2020-02-12 11:43:23.226]Exception from container-launch:
ExitCodeException exitCode=132:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:998)
at org.apache.hadoop.util.Shell.run(Shell.java:884)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1216)
at org.apache.hadoop.yarn.server.nodemanager.DockerContainerExecutor.launchContainer(DockerContainerExecutor.java:312)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.launchContainer(ContainerLaunch.java:443)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:294)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:92)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[2020-02-12 11:43:23.226]
[2020-02-12 11:43:23.228]Container exited with a non-zero exit code 132. Last 40960 bytes of runtime.pai.agg.error :
[PAI_RUNTIME_ERROR_START]
exitcode: 132
trigger: caughtSignal
reason: |
solution: |
originalUserExitCode: 132
dockerStatus:
caughtSignal: SIGILL
caughtException:
matchedUserLogString:
errorLogs:
user: |
f/FrontEnd/Dialog.pm line 76, <> line 1.)
debconf: falling back to frontend: Readline
Cloning into 'models'...
 Memory limited without swap.
1581507802 INFO USER_EXIT_CODE 132
1581507802 ERROR MESSAGE Container exited because of the OS signal
1581507802 ERROR RECEIVE_SIGNAL "SIGILL"
diskCleaner: |
[PAI_RUNTIME_ERROR_END]
[2020-02-12 11:43:23.228]

Here's the job config

protocolVersion: 2
name: admin_1581505909356
type: job
jobRetryCount: 0
prerequisites:

type: dockerimage
uri: openpai/tensorflow-py36-cpu
name: docker_image_0
taskRoles:
Task_role_1:
instances: 1
completion:
minFailedInstances: 1
minSucceededInstances: 1
dockerImage: docker_image_0
resourcePerInstance:
gpu: 0
cpu: 4
memoryMB: 12288
commands:

apt update
apt install -y git
'git clone https://github.com/tensorflow/models'
cd models/research/slim
pip install --upgrade pip
pip3 install contextlib2 Pillow tensorflow==1.14

python download_and_convert_data.py --dataset_name=cifar10
--dataset_dir=/tmp/data

python train_image_classifier.py --dataset_name=cifar10
--dataset_dir=/tmp/data --max_number_of_steps=1000 --clone_on_cpu=True
taskRetryCount: 0
defaults:
virtualCluster: default

Hi, this issue is probably cased by your hardware, e.g. the cpu in your openstack cloud doesn't support AVX instructions. You can refer to related TensorFlow issues.
To confirm, simply change commands to:
pip3 install tensorflow==1.14
python -c "import tensorflow as tf"

You will get "Illegal instruction (core dumped)" as well. It would fail both in Docker or your VM host.
You could also check the log of cat /proc/cpuinfo.
Here're possible solutions:

make your cloud/virtualization support AVX
build tensorflow from source instead of installing by pip

Thanks abuccts! I make my cloud support AVX and the error is gone. But then another error occurs, I tried to add memory from 8GB to 64GB or change anther image(ufoym/deepo:tensorflow-py36-cpu) ,but the new error still exists, would you please take a look? Thx!
Here's the "View Exit Diagnostics"
[Exit Trigger Info]

ExitTriggerMessage: FailedTaskCount 1 has reached MinFailedTaskCount 1 in TaskRole [Task_role_1]
ExitTriggerTaskRole: Task_role_1
ExitTriggerTaskIndex: 0
UserExitCode: 128

--------------------------------------------------------------------------------

[Exit Spec]

code: 255
phrase: CONTAINER_RUNTIME_UNKNOWN_FAILURE
issuer: PAI_RUNTIME
causer: UNKNOWN
type: UNKNOWN_FAILURE
stage: COMPLETING
behavior: UNKNOWN
reaction: RETRY_TO_MAX
reason: Container failed but the failure cannot be recognized by PAI Runtime
repro:
  - User program directly exits with exitcode 1
solution:
  - Check container log and find root cause
  - Wait result from next retry

--------------------------------------------------------------------------------

[Exit Diagnostics]

[2020-02-14 02:49:35.750]Exception from container-launch:
ExitCodeException exitCode=255:
	at org.apache.hadoop.util.Shell.runCommand(Shell.java:998)
	at org.apache.hadoop.util.Shell.run(Shell.java:884)
	at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1216)
	at org.apache.hadoop.yarn.server.nodemanager.DockerContainerExecutor.launchContainer(DockerContainerExecutor.java:312)
	at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.launchContainer(ContainerLaunch.java:443)
	at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:294)
	at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:92)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

[2020-02-14 02:49:35.750]
[2020-02-14 02:49:35.751]Container exited with a non-zero exit code 255. Last 40960 bytes of runtime.pai.agg.error :
[PAI_RUNTIME_ERROR_START]
exitcode: 255
trigger: originalUserExitCode
reason: |
  User process exited with code: 128
solution: |

originalUserExitCode: 128
dockerStatus:
caughtSignal:
caughtException:
matchedUserLogString:
errorLogs:
  user: |

    WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

    WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

    debconf: unable to initialize frontend: Dialog
    debconf: (No usable dialog-like program is installed, so the dialog based frontend cannot be used. at /usr/share/perl5/Debconf/FrontEnd/Dialog.pm line 76, <> line 1.)
    debconf: falling back to frontend: Readline
    Cloning into 'models'...
    error: RPC failed; curl 56 GnuTLS recv error (-9): A TLS packet with unexpected length was received.
    fatal: The remote end hung up unexpectedly
    fatal: early EOF
    fatal: index-pack failed
  runtime: |
<PHONE_NUMBER> ERROR REASON "User process exited with code: 128"
    WARNING: Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.
<PHONE_NUMBER> INFO USER_EXIT_CODE 128
  diskCleaner: |
[PAI_RUNTIME_ERROR_END]

[2020-02-14 02:49:35.752]

And here's the job config:
protocolVersion: 2
name: admin_1581644418489_93135352
type: job
jobRetryCount: 0
prerequisites:
  - type: dockerimage
    uri: openpai/tensorflow-py36-cpu
    name: docker_image_0
taskRoles:
  Task_role_1:
    instances: 1
    completion:
      minFailedInstances: 1
      minSucceededInstances: 1
    dockerImage: docker_image_0
    resourcePerInstance:
      gpu: 0
      cpu: 8
      memoryMB: 65536
    commands:
      - apt update
      - apt install -y git
      - 'git clone https://github.com/tensorflow/models'
      - cd models/research/slim
      - pip install --upgrade pip
      - pip3 install contextlib2 Pillow tensorflow==1.14
      - >-
        python download_and_convert_data.py --dataset_name=cifar10
        --dataset_dir=/tmp/data
      - >-
        python train_image_classifier.py --dataset_name=cifar10
        --dataset_dir=/tmp/data --max_number_of_steps=1000 --clone_on_cpu=True
    taskRetryCount: 0
defaults:
  virtualCluster: default

I still couldn't reproduce it. According to logs you pasted:
Cloning into 'models'...
error: RPC failed; curl 56 GnuTLS recv error (-9): A TLS packet with unexpected length was received.
fatal: The remote end hung up unexpectedly
fatal: early EOF
fatal: index-pack failed

It's a git clone issue, you should search error: RPC failed; curl 56 GnuTLS recv error (-9): A TLS packet with unexpected length was received for that.
Maybe you used a http proxy or customized apt source, or there exists network issue in your cloud.
If retrying the job doesn't work, you could try the following (reference):
apt install -y gnutls-bin
git config --global http.sslVerify false
git config --global http.postBuffer<PHONE_NUMBER>

or find the root cause in your environment.

Hi Abuccts, it works! Thank you for helping me out! The following lines are the magic:
apt install -y gnutls-bin
git config --global http.sslVerify false
git config --global http.postBuffer<PHONE_NUMBER>

@liuc-shaiic Glad it works. I will close this issue, pls feel free to reopen if you have additional questions.


Bug: Ollama connector does not send the tools/agents/functions list

Describe the bug
I am using Semantic Kernel with Ollama and llama 3.2.
When I use the Ollama connector, the functions are not added to http request.
The same code and prompt using Azure OpenAI works correctly.
To Reproduce
OllamaFunctionCalling sample.
Expected behavior
I expect the functions being called.
Platform

OS: Windows 11
IDE: VS2022
Language: C# / .NET 9
Source: 1.28 alpha / preview

Details
I tried the same prompt on GPT and then on Ollama hosted locally.
When I use Ollama I specify the following settings:
KernelFunction[] functions = kernel.Plugins
                .SelectMany(p => p).ToArray();
FunctionChoiceBehavior functionOptions = FunctionChoiceBehavior.Required(functions, true);

OllamaPromptExecutionSettings ollamaPromptExecutionSettings =
    new()
    {
        FunctionChoiceBehavior = functionOptions,
        ModelId = modelName,
        //...
    }

I also tried FunctionChoiceBehavior.Auto() wit no success.
I then passed my own HttpClient with a handler which logs all the http traffic. This is the result:
{"model":"llama3.2","messages":[{"role":"system","content":"","images":null,"tool_calls":null},{"role":"user","content":"What's the weather like in Genoa right now?","images":null,"tool_calls":null}],"options":{"temperature":0.85,"top_p":0.9},"stream":true,"tools":null,"CustomHeaders":{}}

BTW what are the capabilities of this connector? It looks like it does not support multimodality.
In other words, a new prompt contains the image but the http message does not.

Hi @markwallace-microsoft, I will work on this and try to raise PR soon. :)

Any news about that?
Thanks


Bump up openssl-3.0.15

https://www.openssl.org/source/openssl-3.0.15.tar.gz is 404 now. It seems to hit negative CDN cache or some issue. I will retry later.

I found openssl project changed the distribution site:

https://www.openssl.org/source/openssl-3.0.15.tar.gz (404)
https://github.com/openssl/openssl/releases/download/openssl-3.0.15/openssl-3.0.15.tar.gz


Change extension page

Describe your suggested feature
update InariManga extension
Other details
No response
Acknowledgements

[X] I have searched the existing issues and this is a new ticket, NOT a duplicate or related to another open or closed issue.
[X] I have written a short but informative title.
[X] I have updated the app to version 0.16.4.
[X] I will fill out all of the requested information in this form.

Mihon doesn't provide support for 3rd party extensions


SearchAnchor search view close and clear buttons don't have hover tooltips

Steps to reproduce

Add a SearchAnchor to your app
Run the app. (I tried macOS, CanvasKit, and skwasm)
Open the full search view
Hover over the close/back or clear buttons
Notice no tooltips show up like other built-in Material widgets

Expected results
Like other Material widgets, I'd expect these icon buttons to have helpful default tooltips.

When I hover over the close/back button, I'd expect it to say "Close" or "Back" depending on the context.
When I hover over the clear button, I'd expect it to say "Clear search".

This is especially important in this case as the clear button could easily be mistaken as close.
Actual results
When hovering over these icon buttons, no tooltips appear.
Code sample
Code sample
import 'package:flutter/material.dart';

void main() {
  runApp(MyApp());
}

class MyApp extends StatelessWidget {
  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      home: Scaffold(
        body: Center(
          child: SearchAnchor.bar(
            barHintText: 'Search',
            suggestionsBuilder: (context, controller) {
              return [];
            },
          ),
        ),
      ),
    );
  }
}

Screenshots or Video

Screenshots / Video demonstration
Search view close button:

Search view clear button:

Flutter version info
Version
Flutter 3.19.0-3.0.pre.43 • channel main • https://github.com/flutter/flutter
Framework • revision f961fdf2ba (4 hours ago) • 2024-01-10 17:37:07 -0500
Engine • revision a045134c91
Tools • Dart 3.4.0 (build 3.4.0-14.0.dev) • DevTools 2.31.0

Replicable mostly on web and desktop platforms, as mentioned in the original description.

 stable, master flutter doctor -v
[!] Flutter (Channel stable, 3.16.5, on macOS 12.2.1 21D62 darwin-x64, locale
    en-GB)
    • Flutter version 3.16.5 on channel stable at
      /Users/dhs/documents/fluttersdk/flutter
    ! Warning: `flutter` on your path resolves to
      /Users/dhs/Documents/Fluttersdk/flutter/bin/flutter, which is not inside
      your current Flutter SDK checkout at
      /Users/dhs/documents/fluttersdk/flutter. Consider adding
      /Users/dhs/documents/fluttersdk/flutter/bin to the front of your path.
    ! Warning: `dart` on your path resolves to
      /Users/dhs/Documents/Fluttersdk/flutter/bin/dart, which is not inside your
      current Flutter SDK checkout at /Users/dhs/documents/fluttersdk/flutter.
      Consider adding /Users/dhs/documents/fluttersdk/flutter/bin to the front
      of your path.
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision 78666c8dc5 (13 days ago), 2023-12-19 16:14:14 -0800
    • Engine revision 3f3e560236
    • Dart version 3.2.3
    • DevTools version 2.28.4
    • If those were intentional, you can disregard the above warnings; however
      it is recommended to use "git" directly to perform update checks and
      upgrades.

[!] Xcode - develop for iOS and macOS (Xcode 12.3)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    ! Flutter recommends a minimum Xcode version of 13.
      Download the latest version or update via the Mac App Store.
    • CocoaPods version 1.11.2

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] VS Code (version 1.62.0)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.21.0

[✓] Connected device (5 available)
    • SM G975F (mobile)       • RZ8M802WY0X • android-arm64   • Android 11 (API 30)
    • Darshan's iphone (mobile)  • 21150b119064aecc249dfcfe05e259197461ce23 •
      ios            • iOS 14.4.1 18D61
    • iPhone 12 Pro Max (mobile) • A5473606-0213-4FD8-BA16-553433949729     •
      ios            • com.apple.CoreSimulator.SimRuntime.iOS-14-3 (simulator)
    • macOS (desktop)            • macos                                    •
      darwin-x64     • Mac OS X 10.15.4 19E2269 darwin-x64
    • Chrome (web)               • chrome                                   •
      web-javascript • Google Chrome 98.0.4758.80

[✓] HTTP Host Availability
    • All required HTTP hosts are available

! Doctor found issues in 1 category.

[!] Flutter (Channel master, 3.19.0-3.0.pre.50, on macOS 12.2.1 21D62
    darwin-x64, locale en-GB)
    • Flutter version 3.19.0-3.0.pre.50 on channel master at
      /Users/dhs/documents/fluttersdk/flutter
    ! Warning: `flutter` on your path resolves to
      /Users/dhs/Documents/Fluttersdk/flutter/bin/flutter, which is not inside
      your current Flutter SDK checkout at
      /Users/dhs/documents/fluttersdk/flutter. Consider adding
      /Users/dhs/documents/fluttersdk/flutter/bin to the front of your path.
    ! Warning: `dart` on your path resolves to
      /Users/dhs/Documents/Fluttersdk/flutter/bin/dart, which is not inside your
      current Flutter SDK checkout at /Users/dhs/documents/fluttersdk/flutter.
      Consider adding /Users/dhs/documents/fluttersdk/flutter/bin to the front
      of your path.
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision 3e5cad037c (68 minutes ago), 2024-01-10 22:42:23 -0500
    • Engine revision ade9f18fa0
    • Dart version 3.4.0 (build 3.4.0-16.0.dev)
    • DevTools version 2.31.0
    • If those were intentional, you can disregard the above warnings; however
      it is recommended to use "git" directly to perform update checks and
      upgrades.

[!] Android toolchain - develop for Android devices (Android SDK version 30.0.3)
    • Android SDK at /Users/dhs/Library/Android/sdk
    ✗ cmdline-tools component is missing
      Run `path/to/sdkmanager --install "cmdline-tools;latest"`
      See https://developer.android.com/studio/command-line for more details.
    ✗ Android license status unknown.
      Run `flutter doctor --android-licenses` to accept the SDK licenses.
      See https://flutter.dev/docs/get-started/install/macos#android-setup for
      more details.

[✓] Xcode - develop for iOS and macOS (Xcode 13.2.1)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Build 13C100
    • CocoaPods version 1.11.2

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] IntelliJ IDEA Ultimate Edition (version 2021.3.2)
    • IntelliJ at /Applications/IntelliJ IDEA.app
    • Flutter plugin version 65.1.4
    • Dart plugin version 213.7228

[✓] VS Code (version 1.62.0)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.29.0

[✓] Connected device (3 available)
    • Darshan's iphone (mobile) • 21150b119064aecc249dfcfe05e259197461ce23 • ios
      • iOS 15.3.1 19D52
    • macOS (desktop)           • macos                                    •
      darwin-x64     • macOS 12.2.1 21D62 darwin-x64
    • Chrome (web)              • chrome                                   •
      web-javascript • Google Chrome 109.0.5414.119

[✓] Network resources
    • All expected network resources are available.

! Doctor found issues in 1 category.

[!] Xcode - develop for iOS and macOS (Xcode 12.3)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    ! Flutter recommends a minimum Xcode version of 13.
      Download the latest version or update via the Mac App Store.
    • CocoaPods version 1.11.2

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] VS Code (version 1.62.0)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.21.0

[✓] Connected device (5 available)
    • SM G975F (mobile)       • RZ8M802WY0X • android-arm64   • Android 11 (API 30)
    • Darshan's iphone (mobile)  • 21150b119064aecc249dfcfe05e259197461ce23 •
      ios            • iOS 14.4.1 18D61
    • iPhone 12 Pro Max (mobile) • A5473606-0213-4FD8-BA16-553433949729     •
      ios            • com.apple.CoreSimulator.SimRuntime.iOS-14-3 (simulator)
    • macOS (desktop)            • macos                                    •
      darwin-x64     • Mac OS X 10.15.4 19E2269 darwin-x64
    • Chrome (web)               • chrome                                   •
      web-javascript • Google Chrome 98.0.4758.80

[✓] HTTP Host Availability
    • All required HTTP hosts are available

! Doctor found issues in 1 category.


Add resource attribute for number of cores

Area(s)
area:host, area:system
Is your change request related to a problem? Please describe.
The profiling agent that recently has been donated to OTEL by Elastic sends the number of present logical CPU cores as host metadata.
Currently, the field name is specific to Elastic and should be changed it to something fitting for OTEL semantic conventions.
Describe the solution you'd like
Add a new resource attribute like system.cpu.logical.count (or under host.cpu.*).
A more generic approach would be to reflect the CPU topology, similar to what Linux systems present (see https://www.kernel.org/doc/Documentation/ABI/testing/sysfs-devices-system-cpu) or at least keep that in mind when deciding for a name.
Describe alternatives you've considered
Use a field name outside the OTEL semantic conventions (what is done currently). Not ideal regarding future compatibility (may theoretically cause conflicts).
Additional context
No response

We had a long discussion on #99 about whether this should be an attribute or a metric and we decided it would be a metric. You can see https://github.com/open-telemetry/semantic-conventions/pull/99#discussion_r1222742111. I don't think we should represent the same concept as both a metric and a resource attribute.

@mx-psi Thank you for the pointer. I just left a comment (https://github.com/open-telemetry/semantic-conventions/pull/99#discussion_r1683104887). My use case would be profiling, where we use the number of logical CPUs as part of the CO2 and $ cost calculations.
Maybe we can leave this issue open for more opinions.

hey @rockdaboot, I'm not sure if cpu cores can be a resource attribute today since it is potentially mutable (https://github.com/open-telemetry/opentelemetry-specification/pull/2384#issue-1150952467)
but I think this limitation is being worked on by the resources and entities SIG, see https://github.com/open-telemetry/community/blob/main/projects/resources-and-entities.md#problem-3-lack-of-mutable-attributes

Hey @trask, thanks for the links, just my point of view...

I'm not sure if cpu cores can be a resource attribute today since it is potentially mutable (open-telemetry/opentelemetry-specification#2384 (comment))

Well, isn't everything above the hardware level mutable during runtime!? For me, the question is where to draw the line, e.g. why are host.name, host.ip and similar attributes considered unmutable but the number of cores is considered mutable?
We are talking about the host/OS level here, not about single processes. Is seems not to be relevant whether the Java runtime is able to pick up changed CPU values or whether taskset or docker are able to limit CPU resources for single processes (as comments in #2384 suggest).
Let's see how the resources and entities SIG decides :)

We discussed this at KubeCon NA 2024 on 2024-11-14. Now that the resources and entities SIG is pushing the possibility of mutable attributes, this is unblocked and could be added. There is precedent about the same concept being represented as both a resource attribute and a metric (e.g. HTTP request body size) so we can add this if useful regardless of the existance of metrics


feat: Add SystemData implementation for easier fetching of all rigid body storages

This change is

bors r+


Check for boundary condition when skipping blocks

Address https://github.com/pachyderm/python-pachyderm/issues/226
Fairly certain that this is the underlying issue, though I need to find a way to test it with hub.
When a file is uploaded in BSIZE blocks and the same file is read at offsets that are a multiple of BSIZE, GetBlocks will fail. Due to the missing check, GetBlocks tries to read 0 bytes from the block that should be skipped.
The obj read returns an invalid range... "googleapi: got HTTP response code 416 with body: ... The requested range cannot be satisfied." Depending on how the object store handles reads of 0, the results could vary.
GetBlocks also gets called from getFile. It is possible that under certain perfect conditions this bug could explain failed downloads (unexpected EOF).

@brycemcanally could you take a look at this? I'm not quite confident enough to say if this is fix a bug or creating one.

This change seems correct to me. The block ref range is [lower, upper), so when the block size as computed is equal to the offset, we should start in the next block. For example, a lower of 0 and upper of 3 would account for bytes 0, 1, and 2 and the block size would be 3. So, an offset of 3 should start at the first byte in the next block. This fix should also be applied to the GetObjects function since that is probably the actual error source for the user. GetBlocks is used for getting output repo files and GetObjects is used for getting input repo files.

@brycemcanally I made the change to GetObjects as well. I have tested this on hub and it works. I'll look into an easy way to add this test in CI.


smart list

it be nice if we had the ability to set smart list for the media to be added.
like
date: 199* and genre: sitcom
would add all show with the genre sitcom that release date fall in the 1990's
ideally i like to setup this list of what to be added to X channel then have the program generate a guide based off of the smart list. so i can set what can be added to what channel and then i never need to touch it again.
channel 1 play 1990's sitcoms with "date: 199* and genre: sitcom "
channel 2 play  cartoons like cartoon network with "genre: Animated and Children's"
channel 3 play reality tv with "genre: reality"
we could set the list to be genortated on X day at X time. i like to have a guide setup for a week and at the end at midnight it would generate a new week of shows.
things to import would be
https://jellyfin.org/docs/general/server/metadata/nfo
data - release date, air date, month, day, year
genre - Action, Adventure, Animated, Children's, ect https://en.wikipedia.org/wiki/List_of_television_formats_and_genres
Director
Writers
Cast - actor
Studios
plot - biography
tagline
runtime
aspectratio
credits
rating
premiered
tag

I'm fairly certain this ask is covered by https://github.com/chrisbenincasa/tunarr/issues/15?


Working example of VoicePipelineAgent in node?

Pretty basic question...
Is there an example of VoicePipelineAgent that works in node?
The example in the repo does not work out of the box, with API keys and so on. I have no idea how to configure Silero – whether that requires an API key – and the errors are meaningless other than to tell me that the text is empty.
The example has no README. How am I supposed to get started? No documentation for the plugins, nothing.
Was really looking forward to using this library but it's been one headache after another. I don't want to have to implement it manually, but I might have to.

have you tried using the voice-pipeline-agent-node example repository? it should provide you with more information than the example baked into this repository, especially if you use the LiveKit CLI.
API documentation for plugins can be found here, but generally plugins are a black box that plugs into the agent the same way with each plugin. documentation for Agents can be found here.
one of your issues is known and we're working on a fix. i will look into the other when i'm back from break.

Well the sample in the provided repo works well, that has made my day, thank you 🙏
The packages needed updating though,

SyntaxError: The requested module '@livekit/agents' does not provide an export named 'metrics'

Also, which two issues were you referring to?

thank you for the heads up! i will get around to updating the packages there.
by the two issues i mean the two others you have filed in this repository, #233 and #232