Finally task not executed when tasks define "retries"

Expected Behavior
Final tasks are always executed
Actual Behavior
In a pipeline with two parallel tasks where both fail and one of them has "retries=2", final tasks are never executed. Pipeline fails later because of timeout.
Steps to Reproduce the Problem

Create Task:

apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
  name: noop
spec:
  params:
  - name: RETURN_CODE
    default: "0"
  steps:
    - name: sleep
      image: registry.access.redhat.com/ubi8/ubi-minimal
      script: |
        #!/usr/bin/env bash
        set -e

        echo "Will sleep for 2 seconds"
        sleep 2

        if [[ $(params.RETURN_CODE) != "0" ]]; then
          echo "Failure"
        else
          echo "Success"
        fi

        exit $(params.RETURN_CODE)

Create Pipeline

apiVersion: tekton.dev/v1beta1
kind: Pipeline
metadata:
  name: test-finally
spec:
  tasks:
  - name: test1
    taskRef:
      name: noop
    params:
      - name: RETURN_CODE
        value: "1"
    retries: 2
  - name: test2
    taskRef:
      name: noop
    params:
      - name: RETURN_CODE
        value: "1"
    # retries: 2
  finally:
    - name: cleanup
      taskRef:
        name: noop

Create pipelinerun

apiVersion: tekton.dev/v1beta1
kind: PipelineRun
metadata:
  generateName: test-finally-
spec:
  pipelineRef:
    name: test-finally
  timeout: 2m

Additional Info
This pipeline works as expected when any of following changes

Task test2 succeeds
Both task1 and task2 have retries=2 (or any number)
Task test1 has retries=1, retries=0 or retries is omitted

@khrm thank you for the issue report!
in case you haven't seen it, we have a related issue about retries, timeouts and finally tasks in https://github.com/tektoncd/pipeline/issues/4071
will take a closer look at your issue soon
/assign

Thanks @jerop 🙏 There could be an issue with retries rather than scheduling. The controller logs says one task failed but two are incomplete (one dag and one finally) before timing out even though both the dag tasks are done:
{"level":"info","ts":"2021-12-09T18:02:10.867Z","logger":"tekton-pipelines-controller","caller":"pipelinerun/pipelinerun.go:613","msg":"PipelineRun test-finally-2zwhr status is being set to &{Succeeded Unknown  {2021-12-09 18:00:22 +0000 UTC} Running Tasks Completed: 1 (Failed: 1, Cancelled 0), Incomplete: 2, Skipped: 0}","commit":"918ca4f","knative.dev/controller":"github.com.tektoncd.pipeline.pkg.reconciler.pipelinerun.Reconciler","knative.dev/kind":"tekton.dev.PipelineRun","knative.dev/traceid":"442ff111-dea7-4dd8-a576-0074d203a545","knative.dev/key":"default/test-finally-2zwhr"}

....

{"level":"info","ts":"2021-12-09T18:02:11.007Z","logger":"tekton-pipelines-controller.event-broadcaster","caller":"record/event.go:282","msg":"Event(v1.ObjectReference{Kind:\"PipelineRun\", Namespace:\"default\", Name:\"test-finally-2zwhr\", UID:\"532c0fe6-3985-4ad7-beaa-18c07f245785\", APIVersion:\"tekton.dev/v1beta1\", ResourceVersion:\"464290\", FieldPath:\"\"}): type: 'Warning' reason: 'Failed' PipelineRun \"test-finally-2zwhr\" failed to finish within \"2m0s\"","commit":"918ca4f"}

There is something weird going on with retries. The last retry is not attempted when a retry is set for one of the two pipelinetasks (test1) while works fine if retry is set for both the pipelinetasks (test1 and test2).
kubectl get pr
NAME                 SUCCEEDED   REASON               STARTTIME   COMPLETIONTIME
test-finally-gcrdn   False       PipelineRunTimeout   7m15s       5m15s
test-finally-gqj7r   False       Failed               116s        79s

kubectl get all
NAME                                             READY   STATUS      RESTARTS   AGE
pod/test-finally-gcrdn-test1-jz9gt-pod-9vc6z     0/1     Error       0          7m19s
pod/test-finally-gcrdn-test1-jz9gt-pod-gmj9l     0/1     Error       0          7m27s
pod/test-finally-gcrdn-test2-ch922-pod-f2jtt     0/1     Error       0          7m27s
pod/test-finally-gqj7r-cleanup-jvck5-pod-c2qqz   0/1     Completed   0          99s
pod/test-finally-gqj7r-test1-p8kp2-pod-httpp     0/1     Error       0          2m8s
pod/test-finally-gqj7r-test1-p8kp2-pod-qd2k8     0/1     Error       0          111s
pod/test-finally-gqj7r-test1-p8kp2-pod-v7frd     0/1     Error       0          2m
pod/test-finally-gqj7r-test2-2gfj2-pod-89lt7     0/1     Error       0          2m8s
pod/test-finally-gqj7r-test2-2gfj2-pod-9tqcx     0/1     Error       0          118s
pod/test-finally-gqj7r-test2-2gfj2-pod-bcn9h     0/1     Error       0          108s

@pritidesai That retry isn't being attempted because one of the tasks failed. You can remove the finally section in the pipeline and run the pipelinrun.  You will notice that pipelinerun will come to pipeline stopping state.

@pritidesai That retry isn't being attempted because one of the tasks failed. You can remove the finally section in the pipeline and run the pipelinrun. You will notice that pipelinerun will come to pipeline stopping state.

@khrm in the test case we have here, test1 and test2 both should be scheduled at the same time. test1 must not be declared failure unless it finishes all the retries.
I did remove finally section and the run is going in stopping state without attempting the last retry. The stopping state means that the currently executing tasks are allowed to execute and finish and no new tasks are scheduled. This is wrong. The task test1 is not failed until finishes all the retries. There are chances that the task finishes successfully in the last retry and can change the final outcome of the task to success even with first two failed attempts.

@pritidesai Because these tasks are in parallel, failure of task test2 implies that pipelinerun would fail. So we don't need to schedule a retry for task test1 anymore. So declaring test1 as failure seems correct to me.

/assign

So declaring test1 as failure seems correct to me.

Nope, I think it's not correct. test1 had initiated along with test2. test1 should not be declared done without exhausting the number of retires.

@dibyom and I spent some time debugging this issue. We noticed a few things:

A pipeline with a single task with retries works as expected.
The example buggy pipelinerun does not always attempt the same number of retries for task1.
When adjusting the amount of time the tasks sleep (so the two tasks finish at different times), retries for task1 are only attempted until task2 finishes.
Attempting to trigger additional reconcile loops by editing the PR or TR does not fix the problem.

I believe the reason the original example has nondeterministic behavior is because there is a race condition for which task finishes first. This is why we tried having the tasks take different amounts of time, which shows that once task2 finishes, both TRs are marked as failed but the PR logic to update the status of TR1 to condition "unknown" is not being triggered.
The current PR controller behavior is to attempt to schedule the next taskrun, and if a taskrun already exists, to consider it a retry, clear the status, and add retries status. The log line from this block of code is not appearing the same number of times as the number of retries, suggesting to me that the PR controller is not continuing to attempt to schedule task1 for some reason.
As for why the finally task does not get run, the GetFinalTasks method will only return Tasks if the DAG has been completed. However, the PR controller logs show that it does not consider all the Tasks to be completed (e.g. Running Tasks Completed: 1 (Failed: 1, Cancelled 0), Incomplete: 2, Skipped: 0}), which is why I believe the finally task is never triggered and the PR is sort of hanging.

I think I know why this is happening:
Consider the PipelineRun at the state where task2 has completed with failure, and task1 has completed some but not all of its retries. Because task2 has failed, the PipelineRun will be considered "stopping", and the function DAGExecutionQueue will not return any schedulable tasks.
However, the function GetFinalTasks will not return any tasks if any DAG tasks are not yet done. Because task1 is only considered failed if it has no remaining retries, it is not done, and no final tasks are returned.
Therefore, the PR controller doesn't have any tasks in the execution queue, and also does not have any finally tasks ready to execute, so the PipelineRun stalls until it is timed out.

That's a great finding @lbernick 👍
The debate here is whether to terminate task1 (even without exhausting the number of retries) when task2 fails. Failing of task2 is triggering the stopping mode which does not schedule any remaining retry of existing failed tasks and results in hanging pipelineRun until timed out.
We need a resolution on how do we define stopping mode here? In my opinion, stopping mode should allow the failed tasks to exhaust their retries but there was a reservation and argument to not go in that direction.
@afrittoli @jerop @vdemeester appreciate your inputs on defining stopping mode 🙏
We can implement and fix the issue once we have an agreement.


can the scrubbing change from line to a button ?

Can the scrubbing be made into a button instead of a vertical line? I want to be able to get the values as the user touches the graph( not drag on the graph) and keep that value displayed.

I'm not sure exactly what you mean, but you should be able to use a ScrubListener to build whatever UI you want around or even ontop of the SparkView.


Request to add Licence

Hello,
I am Maheshwari from GeekyAnts. We have open source website "Start-React" where we add open source products which we find helpful to the community & also we provide credits to author itself.
Let me know if you are interested to showcase your product in our open source website. If yes, then I request you to add MIT or Apache Licence in your repo.
Looking forward to hear from you.

MIT license added

Hello,
Thanks a lot. We have added your product in Start React
https://startreact.com/ Website.
Best Regards,
Maheshwari Shukla
On Tue, May 28, 2019 at 2:18 PM Andy<EMAIL_ADDRESS>wrote:

MIT license added
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/imandyie/react-native-airbnb-clone/issues/14?email_source=notifications&email_token=AL3JCMJYMG5V4GTBCKSQTGLPXTWUTA5CNFSM4HP3XXC2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODWLNNCA#issuecomment-496424584,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AL3JCMJUGOVUGROWY4RJ4MDPXTWUTANCNFSM4HP3XXCQ
.


Status of Next-Api-Decorators

Next-Api-Decorators package has become a staple of my work project and many of my passion projects. As the title says  - I was wondering what the Development / Project status was for this package? Is it no longer being worked on by it's core development team?
I ask as looking at the last merge was back during the Summer of 2023 (correct me if I'm wrong).

I would love to know too. Missing next-api-decorators in app router A LOT.

@ggurkal ?


[BUG] TypeError: 'int' object is not subscriptable

Describe the bug
I tried to run the gui to produce some images but got an error instead
To Reproduce
Steps to reproduce the behavior:

Open GUI
start a run
See error

Expected behavior
The Run should start, and the images should be created.
Screenshots

Desktop (please complete the following information):

OS: Arch Linux with kernel 5.19.1-arch2-1
GPU and VRAM: RTX 3090, 24GB
CUDA Version: 11.7

Additional context
The problem first occurred with a different error, shown here:
original error
but a reinstallation of NumPy fixed this. I then tested the same settings in a copy of the old GUI, which works fine

This error seems to affect all branches except branch KnoBuddy-rebase, as this has a different error: KeyError: simple_symmetry
I have not yet tested if a slightly older version of prd.py would work without the error, if so, I'll add an edit to this report.
EDIT: older Versions of prd.py also produce the error

Seconding - I just encountered the same thing on windows. I ran a couple of prompts successfully but then started getting this error every time. If it's relevant, the error first appeared for me after I downloaded ViTL16 & RN50x4; they appear to have downloaded successfully but every run I've attempted afterwards has thrown the same error. I was previously simply using ViTB32 & RN50.
I don't know if this is relevant, but the only other setting I can think of that I changed in between runs is CGS; I tried setting it to 18000 from auto along with the new models.
OS: Windows 10 Education, 21H1
GPU/VRAM: RTX 2070 Super, 8 GB (not ideal, I know, but I still prefer it to colab :) )
CUDA/Torchvision: Cuda 11.7 & Torchvision 0.11.3+cu113

Try setting symmetry scale to a schedule. This would match the current value you have but in schedule form:
[20000]*1000

I recently changed this to use a schedule, but the UI might still be treating it as an integer.

Symmetry Scale was changed to a schedule in a recent update to PRD and you must input a script in schedule form. It was a quick fix, I haven't decided the final implementation. Ideally it will accept both schedules and just a plain value.

Also, those other branches are not intended for anyone to use and are for my development purposes only. Obviously it's public, do as you please, but whatever is up there please don't report bugs on those branches.

Try setting symmetry scale to a schedule. This would match the current value you have but in schedule form:
[20000]*1000

I recently changed this to use a schedule, but the UI might still be treating it as an integer.

changing it to this fixed the error


[Question] Does deepspeed support model parallelism via API "PipeModelDataParallelTopology"?

Describe the Question
I just notice code in https://github.com/HuangLK/llama-deepspeed/blob/faedea514b11c18c695e1b2a6adb63b102ef001c/models/llama_pipeline_model.py#L174
It appears that the code is utilizing the PipeModelDataParallelTopology API.
I'm curious to know if it supports model parallelism during training.
I have extensively searched the Internet, documents, and issues, but unfortunately, I have been unable to find an answer to this question. Furthermore, it seems that there are no available documents providing an explanation or usage guide for this particular API.

Hi @chenyaofo , yes the PipeModelDataParallelTopology helps facilitate combining forms of parallelism for training (eg, 3D parallelism for LLMs). It's still up to the provided model to actually implement model parallelism, though. Those topology classes are used to coordinate distributed processes and establish a consistent mapping of ranks between parallel subgroups for the different parallelisms. This is important for situations like computing global norms for gradient clipping.
You'll also find this API used in our repo Megatron-DeepSpeed.


Upgrade to .NET 4.8 breaks API_Caller

Hi,
We recently upgraded our database server with .NET 4.8 Framework in which our API_Caller dlls reside on.   This broke our API_Caller assemblies.
The first error reported was the System.Runtime.Serialization.dll wasn't loaded.  I ran the following SQL from your site:
ALTER  ASSEMBLY [System.Runtime.Serialization]
FROM 'C:\Windows\Microsoft.NET\Framework64\v4.0.30319\System.Runtime.Serialization.dll'
WITH PERMISSION_SET = UNSAFE
Then I get the following error:
An error occurred in the Microsoft .NET Framework while trying to load assembly id 65549. The server may be running out of resources, or the assembly may not be trusted with PERMISSION_SET = EXTERNAL_ACCESS or UNSAFE. Run the query again, or check documentation to see how to solve the assembly trust issues. For more information about this error:
System.IO.FileLoadException: Could not load file or assembly 'api_consumer, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=null' or one of its dependencies. An error relating to security occurred. (Exception from HRESULT: 0x8013150A)
System.IO.FileLoadException:
I tried the following SQL:
ALTER  ASSEMBLY [API_Consumer]
FROM 'C:\APIConsumer_DLL\API_Consumer.dll'
WITH PERMISSION_SET = UNSAFE
And this is the error I get:
ALTER ASSEMBLY failed because the source assembly is, according to MVID, identical to an assembly that is already registered under the name "API_Consumer".
Any ideas on how we can fix this?
Thank you,
Sean

Hi @sramsay335i,
I just saw it. Great to heard that it worked after reinstalling. I'll be updating a new version of this project soon. Where I did implement a table value function to calls the API for GET request. Avoiding to create temporary tables.
Appreciate the feedback.


Busboy does not recognize image/jpeg without filename as files

When using Multer, which wraps around Busboy, the req.files property will populate with all files registered from Busboy.
When using a new software that sends multipart/form-data types, the image/jpeg parts would come into the req.body (fields) not req.files.
Upon tweaking the software to send a filename property with the image part, the images were recognized as files.
Why not add the type image/jpeg as a file type even if it doesn't have the filename property?
I'm happy to make a PR for this issue.

The unfortunate thing is that within the relevant RFC(s) there is no hard differentiator between what is a file and what is a field. This module utilizes the traditional differentiators used by browsers over the past decades, which is either specifying a part content type of application/octet-stream and/or specifying a filename.
I'm not really interested in expanding the part content types used to detect "files" as it would just be arbitrary anyway (and thus not something I'd want to have to maintain) and potentially breaking for many people.
Just have your HTTP client send a filename (even if empty) or use application/octet-stream.


Error in training ssd_mobilenet_v1_pets.config for my own dataset

Hi,
Iam getting this error when i run the command-  python3 train.py --logtostderr --train_dir=training/ --pipeline_config_path=training/ssd_mobilenet_v1_pets.config
/home/shilpa.j/.local/lib/python3.6/site-packages/tensorflow/python/summary/writer/writer.py:386: UserWarning: Attempting to use a closed FileWriter. The operation will be a noop unless the FileWriter is explicitly reopened.
warnings.warn("Attempting to use a closed FileWriter. "
Traceback (most recent call last):
File "train.py", line 184, in
tf.app.run()
File "/home/shilpa.j/.local/lib/python3.6/site-packages/tensorflow/python/platform/app.py", line 40, in run
_run(main=main, argv=argv, flags_parser=_parse_flags_tolerate_undef)
File "/home/shilpa.j/.local/lib/python3.6/site-packages/absl/app.py", line 300, in run
_run_main(main, args)
File "/home/shilpa.j/.local/lib/python3.6/site-packages/absl/app.py", line 251, in _run_main
sys.exit(main(argv))
File "/home/shilpa.j/.local/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py", line 324, in new_func
return func(*args, **kwargs)
File "train.py", line 180, in main
graph_hook_fn=graph_rewriter_fn)
File "/opt/tensorflow/models/research/object_detection/legacy/trainer.py", line 416, in train
saver=saver)
File "/home/shilpa.j/.local/lib/python3.6/site-packages/tensorflow/contrib/slim/python/slim/learning.py", line 790, in train
ignore_live_threads=ignore_live_threads)
File "/home/shilpa.j/.local/lib/python3.6/site-packages/tensorflow/python/training/supervisor.py", line 839, in stop
ignore_live_threads=ignore_live_threads)
File "/home/shilpa.j/.local/lib/python3.6/site-packages/tensorflow/python/training/coordinator.py", line 389, in join
six.reraise(*self._exc_info_to_raise)
File "/home/shilpa.j/.local/lib/python3.6/site-packages/six.py", line 693, in reraise
raise value
File "/home/shilpa.j/.local/lib/python3.6/site-packages/tensorflow/python/training/queue_runner_impl.py", line 257, in _run
enqueue_callable()
File "/home/shilpa.j/.local/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1279, in _single_operation_run
self._call_tf_sessionrun(None, {}, [], target_list, None)
File "/home/shilpa.j/.local/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1429, in _call_tf_sessionrun
run_metadata)
tensorflow.python.framework.errors_impl.InvalidArgumentError: image_size must contain 3 elements[4]
[[{{node cond_1/RandomCropImage/sample_distorted_bounding_box/SampleDistortedBoundingBoxV2}}]]
shilpa.j@Theorem-TrainUP:/opt/tensorflow/models/research/object_detection$

tensorflow.python.framework.errors_impl.InvalidArgumentError: image_size must contain 3 elements[4]
I think you should review your train CSV files CSV files to see if there are any entries with Width and Height as 0 or missing some value.

Please provide details about what platform you are using (operating system, architecture). Also include your TensorFlow version. Also, did you compile from source or install a binary?
If you are unclear what to include see the issue template displayed in the Github new issue template.
Can you please provide full code snippet to reproduce the issue
We ask for this in the issue submission template, because it is really difficult to help without that information. Thanks!

Automatically closing due to lack of recent activity. Please update the issue when new information becomes available, and we will reopen the issue. Thanks!

tensorflow.python.framework.errors_impl.InvalidArgumentError: image_size must contain 3 elements[4]
I think you should review your train CSV files CSV files to see if there are any entries with Width and Height as 0 or missing some value.

Thanks @Arritmic .Yes this has resolved my problem.

This is my system config -
System information
OS Platform and Distribution (e.g., Linux Ubuntu 18.04): Ubuntu
TensorFlow installed from (source or binary):
TensorFlow version: 1.14.0
Python version: 3.5.0
Installed using virtualenv? pip? conda?: virtualenv
GCC/Compiler version (if compiling from source):
CUDA/cuDNN version: Using Linux server 8GB RAM with 4 cored on windows machine
Windows OS config:
processor:i3<EMAIL_ADDRESS>RAM:4GB
System type:64 bit
  GPU model and memory: N/A

I solved it by reviewing csv data for train set and test set for zeros in width column and height column.
Thank you all


What problem does this feature solve?
提供自定义进度显示
What does the proposed API of configuration look like?
和webpack.ProgressPlugin参数一致即可

See:

https://github.com/web-infra-dev/rspack/discussions/4202
https://github.com/web-infra-dev/rspack/pull/4522


Theme color scheme issue

hello i just reinstalled spicetify due to it's missing by itself. but then i just realised that the theme color scheme is bugged after reinstalling, they didn't shown as list as shown below

then i tried to change it from powershell with this command

it restart the spicetify, but when it launched again the theme is still remain the same with wealthy
is there any other way to change the scheme color?

I believe this has to do with you installing through Marketplace, but I am not sure... 🤔 @spicetify/marketplace

@afonsojramos Yup.


Converted "MKV video files" are showing the preview image in the Image window, not the video.

[15:05:31.100][Info] Loading image details from https://danbooru.donmai.us/posts/7922649
[15:05:31.549][Info] Image loaded from the file C:/Users/jelly/Grabber/#Downloads/vid/d6d2d0a2b1c9fa441fd9fa2ef59bd989.mkv


Add DTC

Tool information

Tool name: dtc
Add or update? Add
Desired version: DTC 1.4.7
Approximate size:
If this is an add request:

Brief description of tool: Used during android kernel compilation
URL for tool's homepage: Available as a package for ubuntu

Virtual environments affected

[ ] macOS 10.15
[ x] Ubuntu 16.04 LTS
[ x] Ubuntu 18.04 LTS
[ ] Windows Server 2016 R2
[ ] Windows Server 2019

Can this tool be installed during the build?
No, need root access to install
sudo apt install device-tree-compiler
Are you willing to submit a PR?

The tool can be installed in runtime using the following scripts:
sudo apt install device-tree-compiler

Installation takes about 7 seconds

@TheNotOnly Considering this is a quick install during build, we won't bake it into the virtual-environments image. Thanks for the suggestion.


ETL Accumulation Drops all Node Labels when Labels Differ or are Missing

Some sample queries:
http://adec8d82f7a954efb8616d9820350e32-74ed9b7b0cfc2502.elb.us-east-2.amazonaws.com:9090/model/assets?window=2022-05-29T00%3A00%3A00Z%2C2022-06-08T00%3A00%3A00Z&aggregate=&accumulate=false&filterServices=Kubernetes&filterTypes=Node&filterNames=ip-172-31-6-46.us-west-1.compute.internal
vs
http://adec8d82f7a954efb8616d9820350e32-74ed9b7b0cfc2502.elb.us-east-2.amazonaws.com:9090/model/assets?window=2022-05-29T00%3A00%3A00Z%2C2022-06-08T00%3A00%3A00Z&aggregate=&accumulate=true&filterServices=Kubernetes&filterTypes=Node&filterNames=ip-172-31-6-46.us-west-1.compute.internal
When we accumulate=true, node labels are missing:

  "code": 200,
  "data": [
    {
      "AWS/297945954695/__undefined__/Compute/aws-scale-cluster-2/Node/Kubernetes/i-027f1c6a87fd18e05/ip-172-31-6-46.us-west-1.compute.internal": {
        "type": "Node",
        "properties": {
          "category": "Compute",
          "provider": "AWS",
          "account": "297945954695",
          "service": "Kubernetes",
          "cluster": "aws-scale-cluster-2",
          "name": "ip-172-31-6-46.us-west-1.compute.internal",
          "providerID": "i-027f1c6a87fd18e05"
        },
        "labels": {

        },
        "window": {
          "start": "2022-05-31T00:00:00Z",
          "end": "2022-06-08T00:00:00Z"
        },
        "start": "2022-05-31T00:00:00Z",
        "end": "2022-06-07T15:00:00Z",
        "minutes": 10980.000000,
        "nodeType": "r5.xlarge",
        "cpuCores": 4.000000,
        "ramBytes":<PHONE_NUMBER>8.000000,
        "cpuCoreHours": 732.000000,
        "ramByteHours":<PHONE_NUMBER>624.000000,
        "GPUHours": 0.000000,
        "cpuBreakdown": {
          "idle": 0.9800576211741043,
          "other": 0.0007055137743041098,
          "system": 0.00489822504335257,
          "user": 0.014338640008239286
        },
        "ramBreakdown": {
          "idle": 0.9579879960539572,
          "other": 0,
          "system": 0.0011260823401815468,
          "user": 0.04088592160586116
        },
        "preemptible": 0.000000,
        "discount": 0.000000,
        "cpuCost": 25.035625,
        "gpuCost": 0.000000,
        "gpuCount": 0.000000,
        "ramCost": 26.066575,
        "adjustment": 0.000000,
        "totalCost": 51.102201
      }
    }
  ],
  "message": "ETL: Asset: QueryAsset([2022-05-30T17:52:22+0000, 2022-06-07T17:52:22+0000), []) from ETLStore[1d] 675.158µs [query 505.237µs] [cloud 78.149µs] [aggregate 62.667µs] [accumulate 28.953µs] [stop 152ns]"
}

5/31 to 6/01 for whatever reason has missing labels. All other days have labels.

Are scenarios possible where a kube_node_labels query fails but the ETL succeeds?
Can we override less recent labels with more recent labels on Accumulate if labels differ for the same key?
If we feel that behavior is wrong, maybe override empty labels with existing labels for the same key?
If we still feel that behavior is wrong, this behavior should be documented.

Sounds like this is WAI. The behavior for adding two assets is:  attempt to merge their labels by retaining only those labels that are common to both label sets. (See func (al AssetLabels) Merge(that AssetLabels) AssetLabels.)
The only "problem" I see here is that labels are missing for some window in the range, so when you add anything with that label-less thing, by definition no labels will be common and thus none will be retained.
I cannot recommend that we make any changes here, because the Add function and the Merge function are general to all aggregating and accumulating. So if we make them retain any more labels than they already do (i.e. the complete common set) then we will start applying labels where they are not relevant and clearly do not belong. I hope that is enough explanation, but please let me know if I can say more.
Have you investigated why the labels briefly went missing? That appears to be the only issue here.

To address your four points directly:

Yes. See ClusterNodes function.

// Do not return errors if these fail, but log warnings
resChNodeCPUModeTotal := optionalCtx.QueryAtTime(queryNodeCPUModeTotal, t)
resChNodeRAMSystemPct := optionalCtx.QueryAtTime(queryNodeRAMSystemPct, t)
resChNodeRAMUserPct := optionalCtx.QueryAtTime(queryNodeRAMUserPct, t)
resChLabels := optionalCtx.QueryAtTime(queryLabels, t)

Not easily, and I would not recommend that.
I would not recommend that.
I do not think the behavior is wrong.

Let's focus on #1 then. Can we get an event added when an optional query fails? What about specifically which optional query failed?

This issue is being closed because it may not be relevant to the OpenCost project and appears stale. If you feel this was closed in error, please open a new OpenCost issue with updated details or if it is still relevant for Kubecost please open an issue with Kubecost Support.


Board Background colour selector

As a rainbow-board user I need a green background colour in order to use rainbow-board as a whiteboard overlay in OBS for live annotations during screen casts or video recordings.
Solution:
Option 1: Background colour is changeable in settings.
Option 2: A 'Green Room' theme

On it.

Could you elaborate what you mean by "Green Room Theme"? We can discuss this on Discord if you like, the server link is in the README file.

The board  background colour of the (not yet existing) Greenboard Theme is set to #00FF00 and not white as for Default Light / Dark Whiteboard,  or black for Default Dark / Light Blackboard. (your existing themes)
This way I can record in OBS a window in which rainbow-board is running as a source on top of the screen I am showing, and the background of rainbow-board becomes completely transparent by using a chroma key filter in set to #00FF00.
This way I can interactively 'whiteboard' on top of what I am showing as the main screen in OBS.

Addendum: The icing on the cake would be a slightly darker green for the ruler options ('Set Background').
This is a nice to have only.

I can make a theme but instead of adding it to the app (which will take time), I an send you a "plugin" that will add the theme to your installation if you tell me your operating system.
The darker green ruler will take more time as well.

No worries, didn't expect such a fast response. ;)
Ubuntu 20.04 LTS

Here's the zipped plugin file: green-screen.zip
The file has a folder called green-screen, you will have to put the folder in ~/.config/Rainbow Board/plugins or ~/.config/rainbow-board/plugins, whichever directory exists.
Secondly, you will have to change the file ~/.config/Rainbow Board/settings.json or ~/.config/rainbow-board/settings.json. Find the line "plugins": [] and replace it with "plugins": ["green-screen"].
Relaunch the app after doing this and new themes will appear in the settings.


Add support for Lumen versioning

When creating the factories, the isLaravel8OrAbove version check always returns false for Lumen, as the version string is Lumen (8.2.4) (Laravel Components ^8.0) instead of just 8.2.4.
Would be great to take the Lumen version into account, to generate factory classes instead of methods.

Hi @noplanman!
I initially wasn't really taking Lumen into consideration.
I'll come back to this in the coming days and change version detection to also support it.

I just changed it to return true to generate the files, as it's a one-off for me.
But maybe others would enjoy a built-in fix.


Allow asset specific band metadata

As discussed yesterday, it is useful to allow asset specific band metadata, for example for the new extension "EO for processing and ingestion" (eop), which will add unit, scale and offset as asset specific metadata. This is currently not possible, but I think it is helpful to allow this generally in the EO extension.
This PR is meant to discuss the need and potential solutions, with me preferring this solution. Bases on PR #245 and may potentially be used for issues #243, #242 and the gain/scale, offset, unit addtions for eop, which have no separate issue.
PR Checklist:

[ ] This PR has no breaking changes.
[ ] I have added my changes to the CHANGELOG or a CHANGELOG entry is not required.
[ ] API only: I have run npm run generate-all. to update the generated OpenAPI files.

I'm still confused on why we want to put metadata in the assets section? Any metadata outside of properties breaks the common metadata model. If you put the unit scale and offset values in properties["eo:bands"] what won't you be able to do? I'm pretty uncomfortable with having some metadata in properties and some in assets.

@hgs-truthe01 After thinking about this again for a while, this actually might not be required any longer due to that fact that we changed from dicts to arrays.
But let me first ask you how you would solve this at the moment:
You have a STAC item with two assets, both have bands that share the same names and basically the same characteristics, but differ in two values such as scale and offset. How would you model that? That's basically the use case that came up previously in a discussion with @matthewhanson and @fredliporace.

@hgs-truthe01
The problem as @m-mohr mentions is that you can have values that vary by asset and item.
For example, with Landsat there is a scale and offset given for each band, see here:
https://landsat-pds.s3.amazonaws.com/c1/L8/086/240/LC08_L1GT_086240_20180827_20180827_01_RT/index.html
Scroll down to RADIOMETRIC_RESCALING and you'll see RADIANCE_MULT_BAND_X and RADIANCE_ADD_BAND_X, along with values for converting to REFLECTANCE as well.
With Landsat-8 it's true that these values are constant across all assets, but that was not always the case. Landsat-7 and earlier have values for scaling that change for each asset. I've also seen this occur in other data sources as well, so it's not limited to Landsat.

Another question I think needs to be resolved. Is this a valid item:
{
  "assets" :{
    "B1": {
      "href": "http://landsat-pds.s3.amazonaws.com/L8/153/025/LC81530252014153LGN00/LC81530252014153LGN00_B1.TIF",
      "type": "GeoTIFF",
      "required": true,
      "eo:bands": [
        {
          "index": 0,
          "gsd": 15
        }
      ]
    }
  },

  "eo:bands": [
    {

      "name": "1",
      "common_name": "coastal",
      "gsd": 30.0,
      "accuracy": null,
      "wavelength": 0.44,
      "full_width_half_max": 0.02
    }
  ]
]
}

Why would we put specific eo fields in one place or the other? What happens if they are in both places?

So, the fact that gsd isn't allowed in assets is exactly the "special" rules that I was trying to avoid with the common metadata model.
I think what you are asking for here is essentially a deep merge on eo:bands between item and asset levels. But with rules on what fields go where.
What I would propose is that we don't have special rules and we make use of the commons extension. This might require a small change to commons extension to say that we should use a deep merge when applying common properties though.
collection level:
{
  "properties": {
    "eo:bands": [
        {
          "center_wavelength": 700,
          "gsd": 15,
          ...
        }
      ]
}

item level:
{
  "properties": {
    "c:id":
    "eo:bands": [
        {
          "offset":10,
        }
      ]
}

I think this would solve the use case you are describing, but if there really is asset specific metadata, then I would suggest we need to add assets[0].properties and follow the same rules as the commons from collections to items to assets.

That doesn't quite do it either @hgs-truthe01  because  you could have an two different assets that reference the same spectral band, but with different scales and offsets.
Case in point again is Landsat, which has one set of scale/offset to get the into Reflectance, and and another scale/offset to get the data in Radiance. These could be two different assets but both reference the same spectral band.
In a similar fashion you might have the actual data file which contains RGB and requires scale/offset to get into TOA reflectance, and then a true color image which also contains RGB but doesn't have scale of offset because the values have already been scaled to a byte.
I definitely would like to use behavior like in the Commons extension do be able to handle this. Because sometimes you might have to specify these fields per asset, and other times they will be the same for the same asset across all Items. But what we're talking about really isn't band specific information though, it really needs to be tied to the assets.
I'd like to wait on this until the next release when we work on an asset definition, I don't feel comfortable pushing this into this release because I think there's still plenty to discuss.

What I would propose is that we don't have special rules and we make use of the commons extension. This might require a small change to commons extension to say that we should use a deep merge when applying common properties though.
[...]
I think this would solve the use case you are describing, but if there really is asset specific metadata, then I would suggest we need to add assets[0].properties and follow the same rules as the commons from collections to items to assets.

This sounds reasonable, but also a bit complex. In the end this means that for example you can have a datetime per asset for example and having multiple of them would lead to a daterange in the Item etc.

I'd like to wait on this until the next release when we work on an asset definition, I don't feel comfortable pushing this into this release because I think there's still plenty to discuss.

Sounds good to skip this for 0.6.0 - leads to us changing the bands and assets in multiple versions drastically, but it's a 0.x version, right? ;-)

I'm also on board for a 0.7 discussion on this.

This really isn't band specific information
That says to me we shouldn't put it in an eo:bands key right?

Moved to 0.8.0, as asset definition stuff is going there.

Talked about this in meeting, and discussed how there hasn't seemed to be anyone with a compelling use case that is driving this anymore. We are closing this for now, and can revisit the issue if there are future use cases that come up.


Add option for transforming lots of files in-place

If you want to use Lebab in your project, you'll likely want to run it over more than just one file, in which case you'll need to resort to shell scripting. Doing something like the following:
for file in $(find . -name "*.js"); do
    lebab $file -o $file
done

Having this functionality built into Lebab is an often requested feature (e.g. see #114). Implementing it poses some challenges however:

Which files should we look for? Only *.js? What about *.jsx? Maybe we'll need an option for that.
Should we also look into hidden directories? Perhaps another option.
Should we also follow symlinks? Maybe yet another option.
Transforming lots of files could take time. Maybe we should log some feedback messages?

I think the important part is to make it easy to use Lebab for the most common use case (transforming a directory of *.js files). The special cases could be covered with external scripting like currently.
To keep lebab easily scriptable, the following should still work:
$ cat foo.js | lebab > out.js
$ lebab foo.js > out.js
$ lebab foo.js -o out.js

It also means, that this should not work:
$ lebab scripts/

Instead, there should be a separate option that would communicate the destructive nature of the operation:
$ lebab --replace scripts/
$ lebab --replace script.js
$ lebab --replace 'scripts/*.jsx'

Implemented the --replace option like described above.


Exceptions thrown in SDK

I can't provide steps to reproduce, but, after installing https://pub.dev/packages/catcher, some SDK exceptions are being caught (rendering Catcher useless). This reports, for example, comes from a user (I don't know exactly how to trigger it and it doesn't happen in all of my devices).
I guess this is related with the poor Dart debugging capabilities (for instance, breaking inside a caught exception: https://github.com/Dart-Code/Dart-Code/issues/2670#issuecomment-668877154), but I'm not sure.
This is rendering most of quality control tools useless (in this particular case, a dialog was presented so the user could send me an e-mail... but there is no user code whatsoever in this case)
Error:
PlatformException(error, Permission Denial: get/set setting for user asks to run as user -2 but is calling from uid u0a202; this requires android.permission.INTERACT_ACROSS_USERS_FULL, null)
Stack trace:
#0 StandardMethodCodec.decodeEnvelope (package:flutter/src/services/message_codecs.dart:572)
#1 MethodChannel._invokeMethod (package:flutter/src/services/platform_channel.dart:161)

#2 TextureAndroidViewController._sendCreateMessage (package:flutter/src/services/platform_views.dart:1039)

#3 AndroidViewController.create (package:flutter/src/services/platform_views.dart:746)

#4 RenderAndroidView._sizePlatformView (package:flutter/src/rendering/platform_view.dart:176)

Device parameters:
id: QPFS30.103-43-7
board: def
bootloader: MBM-3.0-def_retail-2ccb26b0b9-200707
brand: motorola
device: def
display: QPFS30.103-43-7
fingerprint: motorola/def_retail/def:10/QPFS30.103-43-7/1a0ca:user/release-keys
hardware: qcom
isPhysicalDevice: true
manufacturer: motorola
model: motorola one hyper
versionBaseOs: motorola/def_retail/def:10/QPF30.103-43/3ed81:user/release-keys
versionCodename: REL
versionIncremental: 1a0ca
versionPreviewSdk: 0
versionRelease: 10
versionSdk: 29
versionSecurityPatch: 2020-07-01

Hi @JCKodel
Catcher is a third-party plugin so this can't be considered an issue with Flutter itself. Please consider opening an issue in the catcher repository.
Closing this for now. Let me know if you disagree and I'll reopen.
Thank you

Hi @JCKodel
Catcher is a third-party plugin so this can't be considered an issue with Flutter itself. Please consider opening an issue in the catcher repository.
Closing this for now. Let me know if you disagree and I'll reopen.
Thank you

This is NOT about Catcher!
This is about Flutter exceptions in the SDK caught by any exception handler package (including Firebase Crashlytics).
This is an internal Flutter exception that can't be fixed by me. Catcher is just an example how bad this is (because it will not log this exception silently)

Thanks for clarifying.
What specific exceptions have you encountered that reflect this issue? Could you also please provide your flutter doctor -v so we know what exactly what SDK version the stack trace is referring to (the line numbers may be incorrect otherwise). Do I understand correctly that the desired behavior here is for exceptions to be uncaught for better debugging?

What specific exceptions have you encountered that reflect this issue?

So far, this one. I'm writing a new version and decided to implement Catcher, at least in Beta, to be able to catch some bugs that doesn't happen on my devices.
The idea is to catch only my exceptions, not Flutter's! I know this is impossible, because Flutter do crash (I have an app with 3 million users and no matter what, exceptions arise in Android SDK and Flutter SDK for a small percentage of devices (there is no relation - is not a specific model, etc.))
But, Catcher shows a popup to the user so he can send the error report to me. If those exceptions come from Flutter, there is nothing I can do to fix them, and those users will always see those popups, no matter what. In other words: nice tools like Catcher is being rendered useless by bugs in some devices.

Do I understand correctly that the desired behavior here is for exceptions to be uncaught for better debugging?

This error was in production (it never happened in debug). There is nothing to do, really. It's an exception. It happened once among 1000 Beta test users. The report is only for you to be aware that there is some kind of bug in some devices and/or environment at that location (I don't know what the error means, but it would be nice if some errors could only just be ignored - perhaps some flag in the global error handling so we know if this is an SDK error or user code error, otherwise there is no point in report them to  me)

Could you also please provide your flutter doctor -v so we know what exactly what SDK version the stack trace is referring to (the line numbers may be incorrect otherwise).

Not really, sorry. That build was built in a Windows 10 machine, using Flutter 1.20.0, 1.20.1 or 1.20.2 (stable channel), I really don't remember and now I'm on a Linux machine with latest 1.20.2, so I can't really tell which version was and the platform itself is different, so, it wouldn't be much useful.

I'm sorry but at this point, I am not sure what can be done here. The exception itself is obscure and happened with a production app with third-party plugins so there is nothing to indicate an issue with flutter itself. If your problem is error handling then this

it would be nice if some errors could only just be ignored - perhaps some flag in the global error handling so we know if this is an SDK error or user code error, otherwise there is no point in report them to me

sounds like a proposal for Catcher, since that is what you are using for reporting.
Finally on ignoring this exception: The exception seems like it occurs during the platform view creation and would cause a fatal crash. Exceptions thrown by the framework can still be used to find and diagnose issues in your app (or usage cases). Just because they weren't thrown by your specific code doesn't mean they don't point to an issue in a use case for your specific code.
It's always helpful to simply google the exception message. In this case, it would lead you to this SO question which contains a good starting point on what the message means and the associated permissions.
Do you have any precise change or new feature you want to propose to flutter itself here?


xcode 上打包编译报No known class method for selector ‘startFromLibraryWithServices:’

Hi @chincegen, thanks for reaching out! I’m not sure what your feature request is, mind if you elaborate your use case scenario?


Deye 12K-SG04LP3 Total PV Power is missing

Many thanks for sharing your project. It works very well for me.
I only noticed that I cannot get the "Total Solar Power" from the logger.
PV1 Power and PV2 Power are available and can be of course added together. But the logger also provides the sum of both.
Maybe the added picture helps:

Thanks.

Hello i have created custom value in ha helpers which combines both values

I did the same but somehow it is showing 0 all the time.
Can you share your config?
`sensor:

platform: template
sensors:
solar_power_total:
friendly_name: "Solar Power Total"
unit_of_measurement: 'kWh'
value_template: '{{ states("entity:sensor.solarman_pv1_power") | float(default=0) + states("entity:sensor.solarman_pv2_power") | float(default=0) }}'`

You can create a "Min / Max" helper where you set "Statistic characteristic" to "Sum". There you can add PV1 and PV2 as sensors to build the sum.

My helpers look different

Min / Max... is the one you want

Select "Sum" at the very bottom:

Yes understood. But in my selection there is no "Sum".

What version of HomeAssistant do you have?

It works on 2023.1.2 and 2023.3.1
Can you try switching to english for testing?

My helpers look different

My helpers look different

Update Home Assistant to the latest version and you will find the “Combine” Helper.


HttpResponseError when deploying model from the tutorial

Hello, I have recently been following azure ml tutorials, and using a free account as of now. When I try to practice the tutorial, I get a constant error at this part of the tutorial, despite restarting multiple times + every other line works fine
After executing the below lines of code:
picking the model to deploy. Here we use the latest version of our registered model
model = ml_client.models.get(name=registered_model_name, version=latest_model_version)
create an online deployment.
blue_deployment = ManagedOnlineDeployment(
name="blue",
endpoint_name=online_endpoint_name,
model=model,
instance_type="Standard_DS3_v2",
instance_count=1,
)
blue_deployment = ml_client.begin_create_or_update(blue_deployment).result()
I encounter a HttpResponseError:
endpoint credit-endpoint-d812de5a exists
data_collector is not a known attribute of class <class 'azure.ai.ml._restclient.v2022_02_01_preview.models._models_py3.ManagedOnlineDeployment'> and will be ignored
I have followed this tutorial exactly as said, with the example notebook copied from the samples.
Any idea why this is happening and how to fix?
thanks
Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 4bee4b9a-ce0f-f98b-3160-d6c4a40ef3ac
Version Independent ID: a0475a9e-47f6-2803-8707-fac54ba898dd
Content: Quickstart: Get started with Azure Machine Learning - Azure Machine Learning
Content Source: articles/machine-learning/tutorial-azure-ml-in-a-day.md
Service: machine-learning
Sub-service: core
GitHub Login: @sdgilley
Microsoft Alias: sgilley

@SamY724
Thanks for your feedback! We will investigate and update as appropriate.

@SamY724
I've delegated this to @sdgilley, a content author, to review and share their valuable insights.

Same issue +1

After checking the traceback from my side, found my issue was caused by the quota limit.
Try increasing the quota or using another instance type might solve your problem.

After checking the traceback from my side, found my issue was caused by the quota limit.
Try increasing the quota or using another instance type might solve your problem.

Good spot, will keep this in mind and maybe upgrade to paid account.

#please-close
@SamY724 - if you still have problems when you have enough quota, feel free to re-open.


Introduce Dimensions::Item#content_uuid

We need a way to identify the same unique item of content.
Until now we have been using base_path to identify an
individual piece of content.
If the base path changes, we will lose the link between the
editions. All metrics collected from GA will be linked to the
individual editions.
The content_uuid will be populated by the Etl process and
will link different versions together.
We can then use this column to join the latest item to
all previous editions, and summarise the metrics across all
editions.
This PR populates the uuid during the Etl process and uses the uuid to join Dimension::Items to the latest version of Dimensions::Item for the index query.

 (5) Index page: show 100 first content items (EPIC 5)

Perhaps a different name for content_uuid might be better because this isn't a UUID according to the spec, and because content_id is also a UUID.
Perhaps something like compound_id, warehouse_item_id, grain_id, internal_page_id?


Delete unused merkle package

Plus minor change to readme.

please make sure UT have passed as well

Please rebase you develop branch and run UT


No internet connection on app

I'm using Orbot, with different android devices and versions I have no internet connection on Nighthawk anymore. I didn't update for a year but all was ok. I got the newest nighthawk version now and the issue is still there. When I try to change server I get some java communication error

Visit https://lightwalletd.com/posts/june2024 to read the latest post and on how it might affect your use of the services.


MSUnmerged: Improve accounting of deletions

Impact of the new feature
MSUnmerged
Is your feature request related to a problem? Please describe.
In the process of testing the service we've got to realize that we are accounting mostly for successful deletions and not logging the failed ones. We also may try to find  way to measure the volume to be released on every iteration, by an additional series of gfal calls to the RSE, once we have the lists for deletions built.
Describe the solution you'd like

Account not only for successful, but also for failed deletions on a per directory basis
implement a series of gfal-ls-like calls to the RSE once the lists of files to be deleted are build or in the process of deletion  (but beware not to consume the file generators too early) so that we may have meaningful assessment of the volume released on the RSE

Describe alternatives you've considered
None
Additional context
Non

This one would benefit from #10797 too.

@todor-ivanov Todor, do you think this is trivial enough and we could get it done in this last 2021 quarter? If so, then we should add it to the Q4 project board.

Thanks, @todor-ivanov . Can you please make sure its status in the board is correct (ToDo? In Progress?)

We need another kind of accounting to judge what kind of workflows are responsible for protecting unmerged directories and if these are really correct. I think the ideal case would be to know the list of workflows which protects the unmerged files in a given RSE and corresponding statistics such as:
RSE: T2_US_Florida

Workflow
Number of protected unmerged files
Total size of the protected unmerged files (GB)
List of protected LFNs
List of protected unmerged files

Workflow-1
16
43
Link to the list of corresponding protected LFNs
Link to the list of corresponding unmerged  files

...
...
...
...
...

This might be too good to be true, but I wanted to set this as the idea goal. It can be brake down into smaller issues and worked out, perhaps. Any comments?
@z4027163 @jenimal  FYI

Hi @haozturk  this is tricky to fetch all that information. One field I can say for sure is impossible to get as of now, this is the Total size. Unfortunately,  the way how the service is constructed right now, we see no volume information when it comes to any data present at the /store/unmerged  branch of the tree. The price for fetching that  info remotely would be thousands of calls to the RSE in order to calculate the size of each file.
The rest is possibly achievable, but creating this would involve doing the reverse mapping of lfn at /store/unmerged -> workflow Name. This reverse mapping would involve calls to several other services like DBS and RequestManager. So again collecting this information for every workflow present at the site at any time, would also has its price. What I was thinking of, when we shortly discussed the issue, was something more like another REST API to the Unmerged service, which would do that reverse mapping on a per call basis. Something like:
https://cmsweb-testbed.cern.ch/ms-unmerged/data/info?rse=T2_US_Florida&lfn=/store/unmerged/RunIISummer20UL16NanoAODv9/QCD_HT300to500_TuneCP5_PSWeights_13TeV-madgraphMLM-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1&detail=True
and the result should be a workflow name.
@amaltaro  please give your thoughts. I will create a new issue to follow there.

This is a very good discussion and definitely something that would be useful to many people. However, we need to make sure that we are not reimplementing work that others have already considered (for instance, unmerged volume is something that I thought it would be implemented in the Consistency Monitoring service).
I would suggest to create a new GH issue with these ideas, where we can keep discussing and defining what the final requirements and expected behavior is. However, I'd suggest not to spend too much investigating possibilities of what the system would support or not. I'd rather first make sure that an unmerged cleanup service is fully functional for all the production RSEs in the system. Then we start looking into enhancements and expanding on new features.


[Data] adding SDSS MaNGA IFU data / support general IFU datasets

Adding SDSS MaNGA IFU data
Add the SDSS-IV MaNGA IFU dataset into the Astropile format.  Also potentially include support for other IFU datasets.
Contacts: @havok2063
Participants:
Goals and deliverable
Building astropile datasets with SDSS MaNGA IFU data.  Once that is done, possibly consider adding other public datasets, like SDSS MaStar stellar template library and / or public JWST IFU data.
Resources needed
familiarity with astropile format,
Detailed description

Tasks:

Decide on IFU datamodel
Write new script to download MaNGA cubes and/or maps
Figure out output format for hugging face
Write processor script to convert input data to output

Tuesday WIP - https://github.com/havok2063/AstroPile_prototype/blob/manga_dlscript/scripts/manga/globus_transfer.py

Wednesday WIP - https://github.com/havok2063/AstroPile_prototype/blob/manga_hdf5/scripts/manga/build_parent_sample.py

Thursday WIP - https://github.com/AstroPile/AstroPile_prototype/pull/33


Update README.md

This change replaces the private link to the repository with the public one.

Thanks


Add color tweaks

Funnily, none of the content on the collapsible header on the second row are clickable. You move to click and it disappears.

Add more colors and fixed the bug with the header :)


Can't install both OpenFoodFacts and OpenBeautyFacts

Summary:
I can't install both OpenFoodFacts and OpenBeautyFacts on the same device. SearchSuggestionProvider is used on both applications and Android seems to dislike that.
Steps to reproduce:

Install OpenBeautyFacts (2.9.8) from F-Droid.
Install OpenFoodFacts (2.9.8) from F-Droid.
Update OpenBeautyFacts to 3.1.0
Try to update OpenFoodFacts to 3.1.0

Or easier:

Install OpenBeautyFacts 3.1.0
Try to install OpenFoodFacts 3.1.0

Add system logs:
04-27 12:58:19.994  1164  1248 W PackageManager: Package couldn't be installed in /data/app/openfoodfacts.github.scrachx.openfood-NXQt09iGA659Yy4-n62pvg==
04-27 12:58:19.994  1164  1248 W PackageManager: com.android.server.pm.PackageManagerException: Can't install because provider name openfoodfacts.github.scrachx.openfood.utils.SearchSuggestionProvider (in package openfoodfacts.github.scrachx.openfood) is already used by openfoodfacts.github.scrachx.openbeauty
04-27 12:58:19.994  1164  1248 W PackageManager:        at com.android.server.pm.PackageManagerService.assertPackageIsValid(PackageManagerService.java:11839)
04-27 12:58:19.994  1164  1248 W PackageManager:        at com.android.server.pm.PackageManagerService.scanPackageNewLI(PackageManagerService.java:10803)
04-27 12:58:19.994  1164  1248 W PackageManager:        at com.android.server.pm.PackageManagerService.scanPackageTracedLI(PackageManagerService.java:10605)
04-27 12:58:19.994  1164  1248 W PackageManager:        at com.android.server.pm.PackageManagerService.replaceNonSystemPackageLIF(PackageManagerService.java:17614)
04-27 12:58:19.994  1164  1248 W PackageManager:        at com.android.server.pm.PackageManagerService.replacePackageLIF(PackageManagerService.java:17561)
04-27 12:58:19.994  1164  1248 W PackageManager:        at com.android.server.pm.PackageManagerService.installPackageLI(PackageManagerService.java:18556)
04-27 12:58:19.994  1164  1248 W PackageManager:        at com.android.server.pm.PackageManagerService.installPackageTracedLI(PackageManagerService.java:18041)
04-27 12:58:19.994  1164  1248 W PackageManager:        at com.android.server.pm.PackageManagerService.access$4000(PackageManagerService.java:460)
04-27 12:58:19.994  1164  1248 W PackageManager:        at com.android.server.pm.PackageManagerService$10.run(PackageManagerService.java:15765)
04-27 12:58:19.994  1164  1248 W PackageManager:        at android.os.Handler.handleCallback(Handler.java:873)
04-27 12:58:19.994  1164  1248 W PackageManager:        at android.os.Handler.dispatchMessage(Handler.java:99)
04-27 12:58:19.994  1164  1248 W PackageManager:        at android.os.Looper.loop(Looper.java:193)
04-27 12:58:19.994  1164  1248 W PackageManager:        at android.os.HandlerThread.run(HandlerThread.java:65)
04-27 12:58:19.994  1164  1248 W PackageManager:        at com.android.server.ServiceThread.run(ServiceThread.java:44)

Expected behavior:
Both applications installed.
Observed behavior:
An error message from Package Manager. The application isn't installed because of an "unknown error".
Device and Android version:
Oneplus 6, on Android 9 (official rom OxygenOS, 9.0.5)

Same for me trying to install the new 3.1.0 from f-droid on a Galaxy S9 with Android 9 with an unknown error and on a Galaxy S6 with LineageOS 14.1 (Android 7.1.2) with "Error -13: The new package failed because it contains a content provider with the same authority as a provider already installed in the system."

Hi. I also tried to install Open Pet Food Facts 3.1.0. The result was similar.

@subhanjansaha

Perhaps a solution like this @deniger @subhanjansaha ?
https://stackoverflow.com/questions/26527722/define-a-string-in-gradle-and-also-use-a-flavor-specific-applicationid

I believe that the fix should be easier as we can use a something like that:
public class SearchSuggestionProvider extends SearchRecentSuggestionsProvider {
  public final static String AUTHORITY = BuildConfig.APPLICATION_ID+".utils.SearchSuggestionProvider";

I had to modify AndroidManifest.xml too:

<provider
            android:authorities="${applicationId}.utils.SearchSuggestionProvider"
            android:name=".utils.SearchSuggestionProvider" />

Was F-Droid notified to trigger them to provide corrected apps?

normally, I've created a release, so it should propagate really soon :-)


Warning message that appears to be unnecessary when reading audio file

Describe the bug
When reading a 10 minute long recording <20190115_Bellingen_Feeding.wav> of bat calls provided by Gary Palmer, a lengthy warning message is produced for every minute saying that the media is "unrecognised" and yet the file appears to be read correctly and a correct output is produced.
Expected behavior
Would not expect an "unrecognised media"  warning when reading a wav file if it ultimately appears to read the recording correctly.
Diagnostics
Below is the warning message when reading the wav file:
2019-08-16T10:17:10.1921253+10:00 [3] WARN  Acoustics.Tools.Audio.FfmpegAudioUtility - Unrecognised media. Extension: wav, Codec Name: pcm_s24le, Codec Long Name: PCM signed 24-bit little-endian, Codec Type: audio Codec Tag String: [1][0][0][0], Codec Tag: 0x0001, Sample Format: s32, Format Name: wav, Format Long Name: WAV / WAVE (Waveform Audio).
The above warning is from the log file. All other output to the log file appears to be normal
The operating system I'm using is: Windows 10.
The version of AnalysisPrograms.exe I'm using is:
The data used can be found:
THe audio file can be found on availae in Gary Palmer folder.
Its name is <20190115_Bellingen_Feeding.wav>

That's pretty interesting. It's technically correct.
I won't have time to look into it yet... I suggest making a copy of your source file and converting to a standard format.


ML.Net model file template cannot be found n Add New Item dialog box

System Information (please complete the following information):

Model Builder or CLI Version:
Visual Studio Version (if applicable):

Describe the bug

On which page of the tutorial did you run into an issue (URL):
Clear description of the problem:

To Reproduce
Steps to reproduce the behavior:

Go to '...'
Click on '....'
Scroll down to '....'
See error

Expected behavior
A clear and concise description of what you expected to happen.
Screenshots
If applicable, add screenshots to help explain your problem.
Additional context
Add any other context about the problem here.

@arnableI79  could you tell us the steps to reproduce the issues for better investigation?

Please checkout this troubleshooting section https://github.com/dotnet/machinelearning-modelbuilder/blob/main/troubleshooting/ModelBuilderTroubleshooting.md#i-cannot-add-machine-learning-to-my-project
@zewditu FYI we have a troubleshooting section for this.

arnabl79 I hope the troubleshooting helps you.


CheckBox is not edge-aligned with other common inputs

Bug type
Component
Component name
MudCheckBox
What happened?
When placed among other controls, the check box is noticeably sticking out.

Expected behavior
The check box should be able to be aligned with the switch, radio, etc.

The problem is that there's no Edge property like the IconButton has, so if you want the others to act like the checkbox, or the checkbox to act like the others, you have no option (AFAIK) except overriding the CSS.
Reproduction link
https://try.mudblazor.com/snippet/wkQoOJbBbqQGUgdQ
Reproduction steps
Observe that the check box is placed further to the right than the others
Relevant log output
No response
Version (bug)
7.0.0-preview3
Version (working)
Not v6.19.1
What browsers are you seeing the problem on?
Edge
On which operating systems are you experiencing the issue?
Windows, Android
Pull Request

[ ] I would like to do a Pull Request

Code of Conduct

[X] I agree to follow this project's Code of Conduct

@henon Is this intentional? Should we do anything about it?

I guess we should change it

The switch looks kind of bad in dark mode

Moved to #9009


Vote crate being stuck in an opening state

the crate is stuck being opened as it has not given me anything or stopped opening and I can't seem to find any way to stop it. it seems to persist even though I log out and back on.

I have the same thing and it never gave me the Dragon Scale that showed up when I opened the crate. mc username: ZeebLUL

Hey @redjeff1! This should now be fixed, can you confirm it for me please? :)


Homebridge-caddy-interlogix setup

Hi I’m very new to homebridge and plug-ins. I don’t have a very good knowledge of all of this yet but am keen to control my alarm system through Apple Home app.
I’m having trouble getting this plugin to work.
It has installed in Homebridge on Raspberey Pi and I have it showing in Home app as a bridge but am unable to use its functions in the Apple home app.
Kind regards.

Hey there,
If you have installed Homebridge properly and you have added it to HomeKit, then you should be able to install the plugin without much hassle. Head over to the Plug-ins section of homebridge-ui-x, search for homebridge-caddx-interlogix and install it. You can use the integrated interface to configure the security system settings as needed (you can find details in the plugin page here in GitHub). Please keep in mind that the plugin requires the NX-595E network module to communicate with, so if your setup is incompatible you will not be able to use it.
Let me know if you still have trouble :)

I’m using the Hills Reliance R8 with comnav installed.

Theoretically it should work. Try to install the plugin and set it up and let me know if it works, otherwise we'll have to do some investigating.

So it has installed ok (I think) but this is where it is showing in Home app.

The reason why you see another bridge installed is because you have set up the plugin in a child bridge (i.e. a different bridge from your main homebridge installation, mostly used for stability reasons or for bypassing device count limitations). Is there a reason for this? If not, you can remove the child bridge settings either from the UI or directly in config.json.
In any case, however, even set up as it is, you should be able to see the security system accessories in HomeKit. Are you sure they don't appear somewhere in the rooms of your configuration?

So I’ve turned off the child bridge and removed from Apple Home. I still can’t see any accessories in HomeKit??
I feel like I’m missing something?
Sent from my iPhone

On 30 Aug 2021, at 21:11, Spyros Smparounis @.***> wrote:

The reason why you see another bridge installed is because you have set up the plugin in a child bridge (i.e. a different bridge from your main homebridge installation, mostly used for stability reasons or for bypassing device count limitations). Is there a reason for this? If not, you can remove the child bridge settings either from the UI or directly in config.json.
In any case, however, even set up as it is, you should be able to see the security system accessories in HomeKit. Are you sure they don't appear somewhere in the rooms of your configuration?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications on the go with GitHub Mobile for iOS or Android.

Also it’s not showing a QR code option to add to HomeKit.
Sent from my iPhone

On 30 Aug 2021, at 23:37, Adam Howell @.***> wrote:
So I’ve turned off the child bridge and removed from Apple Home. I still can’t see any accessories in HomeKit??
I feel like I’m missing something?
Sent from my iPhone

On 30 Aug 2021, at 21:11, Spyros Smparounis @.***> wrote:

The reason why you see another bridge installed is because you have set up the plugin in a child bridge (i.e. a different bridge from your main homebridge installation, mostly used for stability reasons or for bypassing device count limitations). Is there a reason for this? If not, you can remove the child bridge settings either from the UI or directly in config.json.
In any case, however, even set up as it is, you should be able to see the security system accessories in HomeKit. Are you sure they don't appear somewhere in the rooms of your configuration?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications on the go with GitHub Mobile for iOS or Android.

Have you added homebridge to HomeKit? If the server runs properly there should be a QR code in the front page of the web interface; if you are running homebridge manually from the command line the QR code is printed out when the server process starts, as well.

Yes it’s added. I’ve removed and re-added but still no alarm accessories.
Sent from my iPhone

On 31 Aug 2021, at 03:29, Spyros Smparounis @.***> wrote:

Have you added homebridge to HomeKit? If the server runs properly there should be a QR code in the front page of the web interface; if you are running homebridge manually from the command line the QR code is printed out when the server process starts, as well.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications on the go with GitHub Mobile for iOS or Android.

All right; please log in to your Hills web interface and copy/paste here the source of your login page. Also, restart homebridge with the -D switch for debugging and copy/paste the server logs so we can check and see what the issue is.

Could you please send me a screen shot of what you need?
Sent from my iPhone

On 31 Aug 2021, at 06:46, Spyros Smparounis @.***> wrote:

All right; please log in to your Hills web interface and copy/paste here the source of your login page. Also, restart homebridge with the -D switch for debugging and copy/paste the server logs so we can check and see what the issue is.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications on the go with GitHub Mobile for iOS or Android.

Unfortunately I'm away from my development setup, so I'll try to guide you as best I can from here :)
Here are some screenshots to show you what I need you to do in homebridge-ui-x (conveniently taken off the Web, but they illustrate our needs well):

From homebridge-ui-x, open the menu, go into settings->homebridge settings and you will find in there the option to restart homebridge with the debug switch on (-D).  After its reset, go to the main screen of homebridge-ui-x and copy the log contents after the reset and paste them here. Likewise, log in to your Hills web interface (judging by your settings, it should be at <IP_ADDRESS>), and using your browser of choice view the page source (i.e. in Safari the keyboard shortcut for that is Cmd-Opt-I) and copy that and paste it here as well.
Hope this helps.

Thankyou. I don’t have a home computer so I’m doing this all on my iphone. I too am away from my setup (at work) so I’ll check it when I get home tonight.
Thankyou so much for your patience.
Sent from my iPhone

On 31 Aug 2021, at 17:06, Spyros Smparounis @.***> wrote:

Unfortunately I'm away from my development setup, so I'll try to guide you as best I can from here :)
Here are some screenshots to show you what I need you to do in homebridge-ui-x (conveniently taken off the Web, but they illustrate our needs well):
From homebridge-ui-x, open the menu, go into settings->homebridge settings and you will find in there the option to restart homebridge with the debug switch on (-D).  After its reset, go to the main screen of homebridge-ui-x and copy the log contents after the reset and paste them here. Likewise, log in to your Hills web interface (judging by your settings, it should be at <IP_ADDRESS>), and using your browser of choice view the page source (i.e. in Safari the keyboard shortcut for that is Cmd-Opt-I) and copy that and paste it here as well.
Hope this helps.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications on the go with GitHub Mobile for iOS or Android.

No worries :) The only scenario where the plugin won't work with your setup is if your network interface is based on the ZeroWire firmware, as it's not compatible with that. In most other cases, it's just a matter of configuration or some such and we most likely will be able to figure something out. Provided we can make it work, we will be able to look into the relay switch functionality as well.

Ahhh ok I think that may be my problem then…….
Sent from my iPhone

On 31 Aug 2021, at 17:33, Spyros Smparounis @.***> wrote:

No worries :) The only scenario where the plugin won't work with your setup is if your network interface is based on the ZeroWire firmware, as it's not compatible with that. In most other cases, it's just a matter of configuration or some such and we most likely will be able to figure something out. Provided we can make it work, we will be able to look into the relay switch functionality as well.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications on the go with GitHub Mobile for iOS or Android.

We'll check that out; for now, please try to gather the details I have described previously and post them here.

Closing this issue due to inactivity.