Cannot browse Youtube resources (other than searching tracks)

When I browse Youtube in Mopidy, the two available directories "My Youtube artists" and "My Youtube playlists" are empty.
There are errors logged for each.
Browsing "My Youtube artists" fails with the following error being logged:
ERROR    2023-09-02 21:40:09,471 [31:YouTubeBackend-3 (_actor_loop)] mopidy_youtube
  Channel.playlists list_channelplaylists error "Server returned HTTP 400: Bad Request.
Request contains an invalid argument."
ERROR    2023-09-02 21:40:09,471 [31:YouTubeBackend-3 (_actor_loop)] mopidy_youtube
  Cannot load youtube:channel:artists
ERROR    2023-09-02 21:40:09,630 [31:YouTubeBackend-3 (_actor_loop)] mopidy_youtube
  Channel.playlists list_channelplaylists error "'playlists'"
ERROR    2023-09-02 21:40:09,631 [31:Core-11 (_actor_loop)] mopidy.core.library
  YouTubeBackend backend caused an exception.
Traceback (most recent call last):
  File "/usr/lib/python3.10/site-packages/mopidy/core/library.py", line 17, in _backend_error_handling
    yield
  File "/usr/lib/python3.10/site-packages/mopidy/core/library.py", line 114, in _browse
    result = backend.library.browse(uri).get()
  File "/usr/lib/python3.10/site-packages/pykka/_threading.py", line 55, in get
    raise exc_value
  File "/usr/lib/python3.10/site-packages/pykka/_actor.py", line 186, in _actor_loop
    response = self._handle_receive(envelope.message)
  File "/usr/lib/python3.10/site-packages/pykka/_actor.py", line 286, in _handle_receive
    return callee(*message.args, **message.kwargs)
  File "/usr/lib/python3.10/site-packages/cachetools/__init__.py", line 737, in wrapper
    v = func(*args, **kwargs)
  File "/usr/lib/python3.10/site-packages/mopidy_youtube/backend.py", line 195, in browse
    playlists = [
TypeError: 'NoneType' object is not iterable

Browsing "My Youtube playlists" fails with the following error being logged:
ERROR    2023-09-02 21:41:54,304 [31:YouTubeBackend-3 (_actor_loop)] mopidy_youtube
  Cannot load youtube:browse
ERROR    2023-09-02 21:41:54,306 [31:YouTubeBackend-3 (_actor_loop)] mopidy_youtube
  Cannot load youtube:browse
ERROR    2023-09-02 21:41:56,632 [31:YouTubeBackend-3 (_actor_loop)] mopidy_youtube
  Channel.playlists list_channelplaylists error "'playlists'"
ERROR    2023-09-02 21:41:56,632 [31:YouTubeBackend-3 (_actor_loop)] mopidy_youtube
  Cannot load youtube:channel:root

I enabled musicapi_enabled, configured my channel_id and the musicapi_cookiefile as well as youtube_api_key for which I also enabled api_enabled.
I am using

mopidy-youtube 3.7
ytmusicapi 1.2.1

I am not sure what to see there tbh but I would expect to see something and no errors being logged.
Am I using mopidy-youtube the wrong way? Does it work for you?

Also, the mopidy-youtube readme explains that musicapi_browser_authentication_file should be configured to make the ytmusicapi authenticate properly but apparently mopidy-youtube does not support such a configuration key.

I realize the ytmusicapi version I used my be too new since there was a breaking change in its version 0.23.0 apparently.
Also, I tried using version 0.22.0 (and 0.23.0) which logged a slightly different error when trying to browse "My Youtube playlists":
ERROR    2023-09-02 21:53:23,140 [31:YouTubeBackend-3 (_actor_loop)] mopidy_youtube
  Channel.playlists list_channelplaylists error "Server returned HTTP 500: Internal Server Error.
Internal error encountered."
ERROR    2023-09-02 21:53:23,140 [31:YouTubeBackend-3 (_actor_loop)] mopidy_youtube
  Cannot load youtube:channel:root
ERROR    2023-09-02 21:53:23,775 [31:YouTubeBackend-3 (_actor_loop)] mopidy_youtube
  Channel.playlists list_channelplaylists error "Server returned HTTP 500: Internal Server Error.
Internal error encountered."
ERROR    2023-09-02 21:53:23,776 [31:YouTubeBackend-3 (_actor_loop)] mopidy_youtube
  Cannot load youtube:channel:root

@mgoltzsche Just ran into this myself. It's because 3.7 was released in May but the fix was merged in July. Put this in your requriements.txt file instead, and make sure to uninstall the existing version, if any, first.
git+https://github.com/natumbri/mopidy-youtube.git@develop#egg=Mopidy-YouTube

@mgoltzsche also musicapi_cookie is now musicapi_cookiefile and it looks like the musicapi_browser_authentication_file doesn't support using env vars

@daredoes Thanks for pointing that out! Unfortunately, when using the latest commit on the develop branch and letting musicapi_browser_authentication_file point to a browser.json file (generated using ytmusicapi) it also doesn't list Youtube contents within the Mopidy library but logs the same error.


in DeepAREstimator I get error prediction

import pandas as pd
from gluonts.dataset.common import ListDataset
from gluonts.model.deepar import DeepAREstimator
from gluonts.trainer import Trainer

df = pd.read_csv('./data/data.csv', header=0, index_col=0)

training_data = ListDataset([
    {"start": df.index[0], "target": df.value[:"2020-01-01 00:00:00"]}
], freq="1D")

estimator = DeepAREstimator(freq="1D", prediction_length=3, trainer=Trainer(epochs=10))
predictor = estimator.train(training_data=training_data)

test_data = ListDataset(
    [{"start": df.index[0], "target": df.value[:"2020-01-01 00:00:00"]}],
    freq="1D"
)

prediction = next(predictor.predict(test_data))

I want predict 2020-01-01,2020-01-02,2020-01-03
but I get 2019-12-29,2019-12-30,2019-12-31
data.csv

timestamp,value
2019-01-01 00:00:00,NaN
2019-01-02 00:00:00,NaN
2019-01-03 00:00:00,NaN
2019-01-04 00:00:00,NaN
2019-01-05 00:00:00,NaN
2019-01-06 00:00:00,NaN
2019-01-07 00:00:00,NaN
2019-01-08 00:00:00,NaN
2019-01-09 00:00:00,NaN
2019-01-10 00:00:00,8000
2019-01-11 00:00:00,400
…………………………
2019-12-17 00:00:00,7600
2019-12-18 00:00:00,5000
2019-12-19 00:00:00,13600
2019-12-20 00:00:00,10000
2019-12-21 00:00:00,NaN
2019-12-22 00:00:00,NaN
2019-12-23 00:00:00,2600
2019-12-24 00:00:00,10200
2019-12-25 00:00:00,5000
2019-12-26 00:00:00,800
2019-12-27 00:00:00,10800
2019-12-28 00:00:00,NaN
2019-12-29 00:00:00,NaN
2019-12-30 00:00:00,NaN
2019-12-31 00:00:00,3800

@ighack can you make your example self-contained? so that it can be copy-pasted and one can verify actual vs expected output?

For example, you could set "start" and "target" to some dummy values in your snippet, without involving loading from a file.

@ostella
_data = np.random.random(365)

training_data = ListDataset([
    {"start": '2019-01-01 00:00:00', "target": _data}
], freq="1D")

estimator = DeepAREstimator(freq="1D", prediction_length=3, trainer=Trainer(epochs=10))
predictor = estimator.train(training_data=training_data)

test_data = ListDataset(
    [{"start": '2019-01-01 00:00:00', "target": _data}],
    freq="1D"
)

Yes,I can get correct prediction,but why.
I have replace NaN to 1234

I get it
I miss some data

@ighack the loss being optimized is negative log-likelihood, there is no harm in it being negative

@ighack I've adapted your example as follows:
import numpy as np
from gluonts.dataset.common import ListDataset
from gluonts.model.deepar import DeepAREstimator
from gluonts.trainer import Trainer

_data = np.random.random(365)

data = ListDataset([
    {"start": '2019-01-01 00:00:00', "target": _data}
], freq="1D")

estimator = DeepAREstimator(freq="1D", prediction_length=3, trainer=Trainer(epochs=10))
predictor = estimator.train(training_data=data)

for forecast in predictor.predict(data):
    print(forecast.index)

and the output that gets printed is:
DatetimeIndex(['2020-01-01', '2020-01-02', '2020-01-03'], dtype='datetime64[ns]', freq='D')

So it seems to me that the right time index is associated with the forecast, do you agree?

@lostella
I agree with you.
I have modify.

timestamp,value
2019-01-01 00:00:00,0
2019-01-02 00:00:00,0
2019-01-03 00:00:00,0
2019-01-04 00:00:00,0
2019-01-05 00:00:00,0
2019-01-06 00:00:00,0
2019-01-07 00:00:00,0
2019-01-08 00:00:00,0
2019-01-09 00:00:00,0
2019-01-10 00:00:00,8000
2019-01-11 00:00:00,400
……………………
2019-01-27 00:00:00,0
2019-01-28 00:00:00,0
2019-01-29 00:00:00,0
2019-01-30 00:00:00,0
2019-01-31 00:00:00,0
2019-02-01 00:00:00,0
2019-02-02 00:00:00,0
2019-02-03 00:00:00,0
2019-02-04 00:00:00,0
2019-02-05 00:00:00,0
2019-02-06 00:00:00,0
………………
2019-12-05 00:00:00,2000
2019-12-06 00:00:00,9600
2019-12-07 00:00:00,0
2019-12-08 00:00:00,0
2019-12-09 00:00:00,0
2019-12-10 00:00:00,12800
2019-12-11 00:00:00,23600
2019-12-12 00:00:00,3400
2019-12-13 00:00:00,10800
2019-12-14 00:00:00,0
2019-12-15 00:00:00,0
2019-12-16 00:00:00,0
2019-12-17 00:00:00,7600
2019-12-18 00:00:00,5000
2019-12-19 00:00:00,13600
2019-12-20 00:00:00,10000
2019-12-21 00:00:00,0
2019-12-22 00:00:00,0
2019-12-23 00:00:00,2600
2019-12-24 00:00:00,10200
2019-12-25 00:00:00,5000
2019-12-26 00:00:00,800
2019-12-27 00:00:00,10800
2019-12-28 00:00:00,0
2019-12-29 00:00:00,0
2019-12-30 00:00:00,0
2019-12-31 00:00:00,3800

I also get get correct prediction
(Timestamp('2020-01-01 00:00:00', freq='D'), -7453.587890625)
(Timestamp('2020-01-02 00:00:00', freq='D'), -1752.99072265625)
(Timestamp('2020-01-03 00:00:00', freq='D'), -2313.981201171875)
but I get minus value

@ighack sorry, I misunderstood what you meant by negative values.
I think substituting NaNs with zeros is not the right thing to do, as the model will take those zeros as genuine data points instead of recognizing them as missing. Could you try keeping them in the data? For example, as np.nan values

@lostella
zero means we don't sell any goods,so I use zero replace NaN.
I don't  think sale is minus value.
that is incorrect.

@ighack I understand, thanks for the explanation. Then I think the default output distribution (student's t distribution) is not ideal. You may want to try out something for positive count data, like negative binomial:
from gluonts.distribution import NegativeBinomialOutput
from gluonts.model.deepar import DeepAREstimator
from gluonts.trainer import Trainer

estimator = DeepAREstimator(
    freq="1D",
    prediction_length=3,
    distr_output=NegativeBinomialOutput(),
    trainer=Trainer(epochs=10)
)

@lostella
Thank for you help, I get correct result.

@ighack cool! Closing the issue, but feel free to post further questions you may have on the matter


Addes colors with mapped to css custom properties

opc-styles
Added colors with mapped to CSS custom properties
✅ All functions are tested in run-time

Need to add icons in to styles?

@diwanshi  @hybridx  @deshmukhmayur
Do Need to add icons into styles?

I think it's good to have icons into components

@gisumit Can we have support for Red Hat Text and Red Hat Display in the following way

h1, h2, h3, h4, h5, h6 Or all the tags which are used for headings should be Red Hat Display
p, span All the text tags should be Red Hat Text

@gisumit Can we have support for Red Hat Text and Red Hat Display in the following way

h1, h2, h3, h4, h5, h6 Or all the tags which are used for headings should be Red Hat Display
p, span All the text tags should be Red Hat Text

Yeah just added it...

The code is okay. But can we just call it op-styles instead of opc??
Also, are we going to publish this as a npm package? If yes, then the package should be named accordingly. i.e. @one-platform/op-styles

Yeah sure... and updated the name


Colors (rgba)

I wanna create a div with a custom background rgba color how could i ?
first i try this way
RunCSS.configure({
	  colors: {
		smoke: {
			lightest: 	rgba2hexP('rgba(0, 0, 0, 0.1)'),
			lighter: 	rgba2hexP('rgba(0, 0, 0, 0.25)'),
			light: 		rgba2hexP('rgba(0, 0, 0, 0.4)'),
			dark: 		rgba2hexP('rgba(0, 0, 0, 0.6)'),
			darker:		rgba2hexP('rgba(0, 0, 0, 0.75)'),
			darkest: 	rgba2hexP('rgba(0, 0, 0, 0.9)')
		}
	  }
	})

rgba2hexP is a custom function for conversion of string rgba to hex, but fail (8digit hex instead of 6)
so i do it this way
RunCSS.configure({
	  colors: {
		smoke: rgbToHex(0,0,0)
	   }
	})

	<div class="fixed pin z-50 overflow-auto bg-smoke flex " style="--background-color-opacity:0.4;">
		test of content
	</div>

is this the correct way or did i miss something

you can style the background opacity via https://tailwindcss.com/docs/background-opacity
if I have some time I might be able to update runcss to support 8 char hexes

houps my bad i just use and old tuto as a model and it seem to be deprecated


Allow mapping without prefix

I splitted #5 and #6, but #5 is required for this PR.

This PR is pretty old now, but today it would be a big backwards incompatibility for anyone using envconfig without specific envconfig field tags. The project has gotten pretty popular in the last year, not so sure it's a great idea to pull the rug out from users.

I agree with @teepark on this one. We need to require the prefix to remain backwards compatible.


Social Media Icons

Change the Social media icons to black when it starts

Currenly you have to turn it on and off and then it comes back to the original state. Resolve this issue

I want to work on this issue under JWOC. But kindly explain me what I have to do , I m little bit confused.

you need to remove the background color from .socialicon

I want to work on this issue under JWOC. But kindly explain me what I have to do , I m little bit confused.

you need to remove the background color from .socialicon

Assign me this issue.

let @omkxr work on this if he can't do it i will assign it to you


Devise + Omniauth: disable email login when users enable SSO

I have completed the official tutorial to add Omniauth.
Now when some users choose to use "Sign in with Google" I want to disable the normal login with email and password for those users (for enhanced security). Note that I want to disable it only for those specific users and not for the entire application.
Basically I would like to build something like Basecamp, where the users can use the email or the SSO, but not both at the same time.
How can I achieve that? I can't find any Devise configuration that would allow that.

Did you ever figure something out for this? I am the same situation and haven't really come up with anything

@chrisgow Unfortunately, no, I didn't find any solution. It would be really important to have this option for security: if you enable SSO (e.g. with Google) you don't want to have other ways to access your account (e.g. email, password resets, etc.).
And you, did you find any solution?

I'm also facing the same situation because we need to support a phased SSO rollout for an existing local-password app.
I'm just starting to look at things but my plan is:

Clean-up the password related fields.
Prevent any password from working if SSO is enabled.

For 1, once a user links an account I call clean_up_passwords on the user which will clear the password and the password confirmation field:  https://www.rubydoc.info/github/plataformatec/devise/Devise/Models/DatabaseAuthenticatable.
For 2, I'm overriding the valid_password? method to return false if SSO has been enabled. Something like:
# user.rb

  def valid_password?(password)
    return false if sso_enabled?

    super
  end

Not perfect, but seems to be a start. Will update again once I get further along.

ty @krsyoung ! I'm giving you a shoutout on this stackoverflow post
https://stackoverflow.com/questions/72760601/devise-omniauth-disable-email-login-when-users-enable-sso/75787006#75787006

Not sure if this is a problem related to Devise or business application? I faced same issues couple of years ago and I came up hiding password fields in the custom template.
@carlosantoniodasilva wdyt?


Change option values to integers (indices) when using Scale critique

Scale's consensus function requires the value of each option to be "a number".

@yifanmai Thanks! That's a good point and I have made changes accordingly.
Let's focus on the logic of the code and not practicability for this PR, as we might need to significantly refactor it anyways like we discussed offline. (e.g. the point you made on using client, not metrics, to do the mapping was great.)


Remove terra-legacy-theme

We are moving away from using scss variables for theming.
Remove:
https://github.com/cerner/terra-ui.com/blob/master/client/webpack.config.js#L113-L115

Resolved in #25


Sync website directly with project README

The project page (ideally at path: https://maheepk.net/projects/kiwings) should display the contents of README file directly. This way, only one file needs to be updated.
Maybe use a Jekyll plugin to include the file from this repository.

Done! Didn't know I'd write a plugin so fast with zero knowledge of Ruby and build it.
See: https://maheepk.net/projects/kiwings


Add a Nav bar with user sign-in

Adding a Navbar with user signin and adding some features in nav bar .
please assign me this task as i want to contribute under GSSOC'23.

Hi, @Tanusingh02 , you may work on this and please refer to the Contribution guidelines on the readme once.
All the best!


Fixing firewall for Oracle cloud VMs

Fixes https://github.com/angristan/wireguard-install/issues/321

I confirm that it solves issue for Oracle cloud VMs. It will be great if this fix is accepted.

Thanks!


Shift and control keys

Why I can't assign "shift" key to first layout, and "control" to second?
It's very useful and possible to do in Windows (Punto Switcher) and Linux (xgeur), but I can't found software to do that on Mac :(

Found answer in https://github.com/noraesae/kawa/issues/5
Close, please


'#15924 removed extra parenthesis'

Fix Issue #15924
Deleted extra parenthesis.

/assign @kbhawkey

Good catch! Thanks!
/lgtm
/approve

/assign @kbhawkey

Thanks for fixing!
/lgtm

/approve


crash

crash in 1.20.1

Vanilla Permissions 0.2.2 for 1.20.2
wrong support verson on mordrinth

I have removed 1.20 and 1.20.1 as supported versions of the 1.20.2 release.


Properly wrapping help messages

hypervisor subcommand and --verbose option didn't have their help message correctly wrapped, making them hard to read in terminal. This patch addresses them.
fixes: #303

message

The first line is strange, is that bug or some issue during copy and paste? There is
^(help-message+176) 10:47:56
Usage: cou plan hypervisors [options]

Also can we make the help even shorter?  The longest line is 124 characters.

@rgildein ^(help-message+176) 10:47:56  came from my shell setting (FISH). Updated it.
And sure, I can further tidy up the help messages

Output from the latest update of this patch ("before" output can be found in #303):
$ cou plan hypervisors -h
Usage: cou plan hypervisors [options]

Show the steps for upgrading nova-compute machines.
This is possible only if control-plane has been fully upgraded,
otherwise an error will be thrown.

Note that only principal applications colocated with nova-compute units
that support action-managed upgrades are within the scope of this command.
Other principal applications (e.g. ceph-osd) and subordinates
can be upgraded via the data-plane subcommand.

Options:
  -h, --help            Show this help message and exit.
  --model MODEL_NAME    Set the model to operate on.
                        If not set, the currently active Juju model will be used.
  --backup, --no-backup
                        Include database backup step before cloud upgrade.
                        Default to enabling database backup.
  --force               Force the plan/upgrade of non-empty hypervisors.
  --verbose, -v         Increase logging verbosity in STDOUT.
                        Multiple 'v's yield progressively more detail (up to 4).
                        Note that by default the logfile will include standard logs
                        from juju and websockets, as well as debug logs from all other
                        modules. To also include the debug level logs from juju and
                        websockets modules, use the maximum verbosity.
  --quiet, -q           Disable output in STDOUT.
  --machine MACHINES, -m MACHINES
                        Specify machine id(s) to upgrade.
                        This option accepts a single machine id as well as a stringified
                        comma-separated list of ids, and can be repeated multiple times.
  --availability-zone AVAILABILITY_ZONES, --az AVAILABILITY_ZONES
                        Specify availability zone(s) to upgrade.
                        This option accepts a single availability zone as well as a
                        stringified comma-separated list of AZs, and can be repeated
                        multiple times.


Refactor/conventional names

use conventional object names.
closes #80
THIS IS A BREAKING CHANGE

this job should pass once #96 is merged


Workspace wildcard member should only match directory

I have a project with the form
service/
   README.md
   service_a/
     pyproject.toml
     src/
   service_b/
     pyproject.toml
     src/

with
members = [
  "services/*"
]

I get
 uv sync
error: failed to read from file `/####/services/README.md/pyproject.toml`
  Caused by: Not a directory (os error 20)

A simple fix would be to filter out files.
It also complains if any subfolder doesn't contain a pyproject.toml. Would it be better if it would ignore them?
Being able to recursively search for pyproject.toml would also be nice.

That's funny, I just added #5724.

It also complains if any subfolder doesn't contain a pyproject.toml. Would it be better if it would ignore them?

You should mark any such directories as excluded via workspace.exclude.

I know about exclude, but I imagined it was to explicitly exclude a folder with a pyproject.toml while not erroring out of folder not containing any.
I can see it both ways..

Yeah, Cargo rejects directories that are listed as members but aren't valid packages, so I think we followed that precedent.


Multiple windows pop up

Hi, I am currently using a Macbook Air (macOS 11.2.1), default terminal with zsh and ST3 (Build 3211).
Steps to reproduce:
(Multiple terminal windows popup at once)
After installing terminal, press command + shift + t.
Two terminal windows pop up together, one in my home directory, another one in the project directory.

It does not work as expected.

Update: No matter you use command + shift + t, command + option + shift + t, or using "Terminal: Open" in Command Palette gives the same result.

Same for me....
I am using MacBook Pro, iTerm2 with zsh.
I was getting 2 windows, one with project directory and the other with home directory(~/).
I never faced this issue on my linux machine.

Issue resolved in iTerm2 by changing the preferences

Change the 'window restoration policy'  to 'only restore hotkey window'.
But when launching the terminal outside the sublime, only the menubar is shown as shown below.

Dont worry we can get terminal by clicking the terminal icon in the dock or simply by pressing 'Command+T'


Fix #183 Delete Setting

Sorting issue Delete Setting #183

Correct me if I'm wrong, but this seems to be working in the current master-branch.

Hey @marktopper , I was checked with master now, it's not updated.
I messed around with my clone rep and this PR has all those commits. The commit related to this issue is this one: https://github.com/the-control-group/voyager/pull/184/commits/0ff92187fca0cc1f6b4668976cd211d2cd6f306d.
btw, nice job for managing voyager ;)

Just tested it, and this makes the job.
/merging


BZ1241414: Ability to set default locale of Business Central

See https://bugzilla.redhat.com/show_bug.cgi?id=1241414

https://github.com/droolsjbpm/drools-wb/commit/25503d511


question about the evaluation result on middlebury datasets

First of all, thank you for your excellent work. However, I have a doubt that I directly use the model on the Middlebury dataset provided by you for testing on the Middlebury dataset，but the performance is poor. So do I need to do any post-processing operations, such as scale, when generating PFM predictions？

First of all, thank you for your excellent work. However, I have a doubt that I directly use the model on the Middlebury dataset provided by you for testing on the Middlebury dataset，but the performance is poor. So do I need to do any post-processing operations, such as scale, when generating PFM predictions？


print() not printing to console on external ios device

I recently upgraded my flutter, and now print() statements to not execute when running on external ios device. They were working previously, before I upgraded, but now that I have upgraded they do not work. Is there any advice? I tried flutter clean but this didn't work. Any help is appreciated

@hossam-zaki,
There's similar open issue describing your case, 44718.
Please follow-up there for updates and any questions.
Closing this as duplicate.


loader.py np.int

As for compatibility with Windows, it would be great to change np.int in loader.py into np.int32 or np.int64
I have to change this manually after installing via. "pip install ." in order to use kgbench inside Windows.
On the other hand: Windows wsl and Linux have no issues with np.int

I've change everything to np.int32. This should be what np.int represents when it works (if it is used in a dtype argument).
I don't have access to a windows machine to test on, so do re-open if this doesn't work after all.
e10ac73d98d260dc4972539053db9d911b42ff0b

Just checked the change on a fresh python env in windows. Seems to work as expected.
Thanks for the quick fix!


Update common_pitfalls.md

Hi! Just a small grammar correction that serves as my introductory PR :)

Awesome, thank you! :)


Running IREE dispatch profiler in CI

[ ] Running IREE dispatch profiler in CI to ensure the tools functional health
[ ] Using IREE dispatch profiler in CI to achieve nightly nightly benchmarking runs, and then pre-submit regression tests.

Steps to execute:

Move the dispatch profiler from iree/experimental to iree/benchmark
[Waiting for concrete step 2. with examples]

The README.md here describes some basic command lines.
Can someone describe steps to run the below sequence of command lines as a part of CI?
$ python3 ../iree/experimental/dispatch_profiler/generator.py
$ python3 ../iree/experimental/dispatch_profiler/compile.py
$ python3 ../iree/experimental/dispatch_profiler/profiler.py --dispatches=matmul_3456x1024x2048_f16t_f16t_f16t_tile_config_128x128_32x5_tensorcore_mmasync,matmul_3456x1024x2048_f32t_f32t_f32t_tile_config_128x128_16x5_tensorcore_mmasync

Then what should be

Can someone describe steps to run the below sequence of command lines as a part of CI?

@GMNGeoffrey @ScottTodd do you have any suggestions on how to run IREE Dispatch Profiler for functional testing as an IREE pre-submit for a bounded number of cases? It would need to run on GPU instances.

Step 1 for running on presubmit is to promote the code out of experimental/.
Tests should be authored using iree_py_test (or iree_native_test, or iree_cc_test), then you won't need to do anything special to get the tests included in our CI builds. Tests requiring physical GPUs should be more carefully evaluated, but they can be added using some labels.
I'd also be careful about test sizes - try to keep the total test time under 2 minutes.

Thanks, @ScottTodd. I think IREE Dispatch Profiler can meet the requirements (move out of experimental/, have bounded test run times, have reasonable stable deps). @manishucsd, we should plan on implementing this as an iree_py_test. I recommend you do this yourself (hopefully < 1 day?), so that you become familiar with how to create these tests.

Step 1 for running on presubmit is to promote the code out of experimental/.

What is a good pace for IREE dispatch profiler outside of experimental/?

benchmarks/ perhaps?

Haven't read all history, but +1 to Scott. I'd also recommend that you test this as a library and test interesting functions with unit tests. We could have some smoke tests that invoke the tool via a shell script or something, but mostly this should be tested with normal python testing facilities

Uh, why are we using a profiler tool to test functional correctness? I thought the goal was to test the tool itself and make sure it continued working

Running IREE dispatch profiler in CI to ensure the tools functional health

But also my point was that these should be normal Python tests using the unittest module, rather than custom thing invoking a binary. The entry point function for the CLI should be directly testable

Uh, why are we using a profiler tool to test functional correctness?

Massive +1. Keeping this simple and focused should be the goal.

I'm a bit out of the loop on how the dispatch profiler works, but my general guidance on adding new benchmarks would be to generate programs (MLIR files using dialects that IREE can import/compile) and then plug those programs into our existing benchmark suites.

https://github.com/openxla/iree/tree/main/build_tools/python/e2e_test_artifacts
https://github.com/openxla/iree/tree/main/tests/e2e/test_artifacts

There's been a separate thread of work/ideas (not a literal email thread / discussion) around running correctness tests on every program we benchmark. This could feed into that too...
Also, if these are microbenchmarks, we'd want to ensure the numbers they produce are stable, which might be more difficult than for full program benchmarks.

Uh, why are we using a profiler tool to test functional correctness?

Massive +1. Keeping this simple and focused should be the goal.

A large part of the IREE Dispatch Profiler as implemented is a generator for dispatch regions and golden inputs/outputs. @manishucsd has added functionality to specify sweeps over ops, ranges etc. The tool also verifies IREE outputs match the golden outputs, although this does not necessarily need to be leveraged in CI-based correctness testing.
I agree there are open questions about whether we should do this, and how it intersects with existing functional testing. I may have been over-eager in suggesting we add this to testing CI, without working through those. We'll circle back.

Uh, why are we using a profiler tool to test functional correctness? I thought the goal was to test the tool itself and make sure it continued working

Currently, we need functional correctness tests for batch_matmul and matmul with split-k. We are also looking to add several new features in the matmul codegen such as supporting unaligned shapes, split-k on unaligned reduction dimension, and new data types (BF16). For the new features, we need simple tool achieve functional verification and performance reporting.
IREE dispatch profiler provides Python-only solution to:

generate mlir dispatches
create/write reference implementation using numpy to produce golden inputs/outputs
runs verification of the generated dispatches against the golden when enabled (--verification-enabled=<true|false>)
runs profiling for the generated mlir dispatch (--profiling-enabled=<true|false>)
(Please find the dispatch_profiler/README.md that describes IREE dispatch profiler capabilities as of today).

Ideally, verification would use iree-run-module or iree-run-trace with --expected_output. We support multiple platforms where the Python interpreter cannot run, so using native (C/C++) tools is preferred. (We've already reinvented this wheel several times... would be good to not do it yet again)

Ideally, verification would use iree-run-module or iree-run-trace with --expected_output.

IREE dispatch profiler uses all of the above features to achieve its goals.
$ python3 ../iree/experimental/dispatch_profiler/profiler.py --dispatches=matmul_3456x1024x2048_f16t_f16t_f16t_tile_config_128x256_32x3_tensorcore_mmasync --verification-enabled=true --profiling-enabled=true --verbose
Skipping compilation of operation: generated/linalg/matmul/matmul_3456x1024x2048_f16t_f16t_f16t/matmul_3456x1024x2048_f16t_f16t_f16t_verify.vmfb since it already exists.
[Verification] tools/iree-run-module --module=generated/linalg/matmul/matmul_3456x1024x2048_f16t_f16t_f16t/matmul_3456x1024x2048_f16t_f16t_f16t_verify.vmfb --device=cuda --function=matmul_3456x1024x2048_f16t_f16t_f16t_tile_config_128x256_32x3_tensorcore_mmasync<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>EXEC @matmul_3456x1024x2048_f16t_f16t_f16t_tile_config_128x256_32x3_tensorcore_mmasync
[SUCCESS] all function outputs matched their expected values.

Skipping compilation of operation: generated/linalg/matmul/matmul_3456x1024x2048_f16t_f16t_f16t/matmul_3456x1024x2048_f16t_f16t_f16t_profile.vmfb since it already exists.
[Profiling] tools/iree-benchmark-module --module=generated/linalg/matmul/matmul_3456x1024x2048_f16t_f16t_f16t/matmul_3456x1024x2048_f16t_f16t_f16t_profile.vmfb --device=cuda --benchmark_repetitions=5 --batch_size=100 --function=matmul_3456x1024x2048_f16t_f16t_f16t_tile_config_128x256_32x3_tensorcore_mmasync --input=3456x2048xf16 --input=2048x1024xf16
----------------------------------------------------------------
Dispatch      : matmul_3456x1024x2048_f16t_f16t_f16t_tile_config_128x256_32x3_tensorcore_mmasync
Provider      : IREE Codegen
OpKind        : OperationKind.Matmul
Operation     : matmul_3456x1024x2048_f16t_f16t_f16t
Configuration : tile_config_128x256_32x3_tensorcore_mmasync
Arguments     : --batch_count=1 --m=3456 --n=1024 --k=2048 --lhs=f16t --rhs=f16t --result=f16t
                --split_k_mode=N/A --split_k_slices=N/A
Verification  : SUCCESS
Runtime(ms)   : 0.062
GFLOPs        : 233798.62

Based on the discussion from IREE dev weekly, please provide an example of similar enablement in the CI. I understand that it won't be a one-to-one mapping for dispatch_profiler use case. IIUC, we are planning on a flow which already exists. Thus, I am hoping the flow has a few examples that would be good starting point to look at.

Can you clarify the scope of this testing and profiling a bit? How many individual program runs are you interested in testing/profiling? How are those program runs organized? How long do those take to test correctness and to benchmark?
We'd want a different style of CI integration for running 10-100 variations, 100-10000 variations, and 10000+ variations.

For the sake of functional testing, is there any reason why we can't just have IREE Dispatch Profiler's generator conform to the same output as https://github.com/openxla/iree/blob/b33c37537d5278019aa5b5865197b0b3f2e121a3/tests/e2e/matmul/generate_e2e_matmul_tests.py and use it in the same manner, as an additional test case generator?
(This is putting aside benchmarking, which I think we should leave as a follow-up, because that's a larger problem, and one that we don't have the right people involved in this thread to figure out.)

Folks, we've got a testing deficit on a bunch of this stuff, and we're going to need to work together to close on it. I haven't reviewed this whole thread or the implications. I will and will help drive a recommendation, but I wanted to set the expectation that we will need to find a way here.

I think the naming of Dispatch Profiler might be causing confusion.  Think of this more as an extensible set of scripts that can run a growing set of workloads in various configurable ways.   It should subsume mmperf, for instance.  What it does and how we use it will evolve, and I ask for some patience in the meantime, but what I was after was to first get this landed so that

we can start getting matmul (and other op) sweeps
have a simple place for others to add more tests
have it also be easily-runnable by folks locally

my current target for near-term CI integration is:

(optionall) get some of its functional testing coverage and goodness integrated into pre-submits. We should answer the question of how this relates to the current matmul tests, and make sure we incorporate lessons learned from those tests. I think a simple integration that follows the model of adding IREE Dispatch Profiler as an additional generator should work here?
Run a defined set of fixed IREE Dispatch Profiler runs nightly (or twice-daily) as we currently do mmperf. This is largely independent of the IREE CI. But that data can be ingested into analytics and dashboards. You don't get pre-submit testing, but it gets us pretty far. We can re-visit pre-submit testing and a deeper benchmark infa integration after this.

I think a simple integration that follows the model of adding IREE Dispatch Profiler as an additional generator should work here?

I connected with @benvanik on this this morning. For functional testing, which we expect to have on the order of O(10000) calls across a number of invocations of dispatch profiler, we want to bottom out on the IREE trace runner instead of iree-benchmark-module (as the current e2e matmul tests do). The current e2e matmul tests use https://github.com/openxla/iree/blob/main/tools/iree-e2e-matmul-test.c, which has specialized handling for verifying matmul calls against expected results. @benvanik and I propose creating an iree-check-trace.c, which would be a general utility for validating invocation results in a trace using gTest. IREE Dispatch Profiler can export its invocations w/ golden inputs/outputs in the trace, and have the existing test infra + iree-check-trace.c execute the validation efficiently. I'll start implementating this tomorrow, May 12.

We've wanted something like that for a while. It's the natural convergence of iree-run-trace and check test. I'd love to see it replace the latter eventually, which would require an ergonomic way to write a test containing a runtime trace spec and an uncompiled model to be compiled on the host (not jit compiled on the test device)

The traces can take modules on stdin today or can include relative file paths so that the generator tool can produce the yaml indicating calls/inputs/outputs and a .mlir source file, the mlir can be compiled into many different vmfbs (different flags, backends, etc), and the same yaml can be used across all of them.

Yeah mostly I want it to be possible to write a test in one file (that's a nice feature of lit tests) where I specify the IR as well the expected pattern of calls. Except that I don't write code anymore :cry: so I guess I really just mean that I think that would be a nice interface for others. More importantly from an infra perspective, any compilation needs to be separable from execution, but the generator design gives us that.

Based on feedback, I prioritized running dispatch profiler in a Github Workflow regularly (as opposed to starting with pre-submit testing). See details in #13629.

@mariecwhite who is looking into helping out here.


[BUG] RKE2, Edit Config, Advanced Menu blank

Rancher Server Setup

Rancher version: v2.6.8
Installation option (Docker install/Helm Chart): docker

If Helm Chart, Kubernetes Cluster and version (RKE1, RKE2, k3s, EKS, etc):

Proxy/Cert Details: Self

Information about the Cluster

Kubernetes version: v1.24.2
Cluster Type (Local/Downstream): local

If downstream, what type of cluster? (Custom/Imported or specify provider for Hosted/Infrastructure Provider):

User Information

What is the role of the user logged in? Admin/Cluster

If custom, define the set of permissions:

Describe the bug

Advanced menu is blank when attempting to edit config of an existing rke2 cluster.
To Reproduce

Select existing RKE2 cluster, and Edit config
Then go to Advanced
Result
It is blank
Expected Result

To have access to the advanced menu
Screenshots

Additional context

I have not updated recently.
There are no console logs or network 404's or anything of the like.
I do not see anything in the Rancher logs.
I have tested with Chrome and Firefox.
I do not have anything else other than RKE2 and imported clusters, so I do not know if this is happening with RKE.
When I go to create a new RKE2 cluster, advanced menu is there

this is not reproducible on v2.6.9 nor v2.7.0 on custom or node driver rke1, rke2 clusters. Could you please try to upgrade?
Closing as you filed a duplicate in dashboard as well, linked above


Nme webgpu3

More nodes!

Please make sure to label your PR with "bug", "new feature" or "breaking change" label(s).
To prevent this PR from going to the changelog marked it with the "skip changelog" label.

Snapshot stored with reference name:
refs/pull/15123/merge
Test environment:
https://babylonsnapshots.z22.web.core.windows.net/refs/pull/15123/merge/index.html
To test a playground add it to the URL, for example:
https://babylonsnapshots.z22.web.core.windows.net/refs/pull/15123/merge/index.html#WGZLGJ#4600
Links to test babylon tools with this snapshot:
https://playground.babylonjs.com/?snapshot=refs/pull/15123/merge
https://sandbox.babylonjs.com/?snapshot=refs/pull/15123/merge
https://gui.babylonjs.com/?snapshot=refs/pull/15123/merge
https://nme.babylonjs.com/?snapshot=refs/pull/15123/merge
To test the snapshot in the playground with a playground ID add it after the snapshot query string:
https://playground.babylonjs.com/?snapshot=refs/pull/15123/merge#BCU1XR#0

WebGL2 visualization test reporter:
https://babylonsnapshots.z22.web.core.windows.net/refs/pull/15123/merge/testResults/webgl2playwright/index.html

Visualization tests for WebGPU (Experimental)
Important - these might fail sporadically. This is an optional test.
https://babylonsnapshots.z22.web.core.windows.net/refs/pull/15123/merge/testResults/webgpuplaywright/index.html


[BUG] Use a stable tag for the quay.io/ceph/ceph:v16.2.7 image

Describe the bug
Recently, after the release of epicli 2.0.0, the quay.io/ceph/ceph:v16.2.7 image tag has been updated, thus its checksum has changed, which resulted in the installation failure while downloading requirements.
As a workaround, the checksum has been updated and the allow_mismatch flag has been added.
We need to investigate why the patch version has been updated and do something to prevent it from happening in the future.
E.g. use an image with a specific release date, e.g.
v16.2.7-20220510
Worth checking out for all the other rook images.
It is also worth checking if the following information about the quay.io repository affects us:
On June 15, 2022, Repository Signing and App Registry will be removed from quay.io. Users should migrate to use alternatives which are now natively supported (Cosign & Helm respectively)
Expected behavior
A given tag should not change over time.
Environment

Cloud provider: [all]
OS: [all]

epicli version: 2.0.0, 2.0.1dev
Additional context
Add any other context about the problem here.

DoD checklist

Changelog

[ ] updated
[ ] not needed

COMPONENTS.md

[ ] updated
[ ] not needed

Schema

[ ] updated
[ ] not needed

Backport tasks

[ ] created
[ ] not needed

Documentation

[ ] added
[ ] updated
[ ] not needed

[ ] Feature has automated tests
[ ] Automated tests passed (QA pipelines)

[ ] apply
[ ] upgrade
[ ] backup/restore

[ ] Idempotency tested
[ ] All conversations in PR resolved

Regarding information Repository Signing and App Registry will be removed from quay.io. Users should migrate to use alternatives which are now natively supported (Cosign & Helm respectively):

its affecting only repo signing and feature appr which are not related to image registry used for ceph images
please see: Moving On From Applications and Legacy Signing on Quay.io for more information

As described in repository information under https://quay.io/repository/ceph/ceph:
new version with a different build date will be built when there is a base image update - therefore when there is an image update a same tag is assigned to the new build, which caused checksum mismatch.
To avoid this kind of situation it is recommended to use tag suffixed with an 8-digit build date in the form YYYMMDD - note from information: These tags are intended for use when precise control over image upgrade is desired and are recommended for production use.

Investigation of other ceph/rook images used in Epiphany:

only ceph/ceph image is described by a tag with build date suffix which gives additional security to have stable tags, all other images doesn't provide this option

See ticket https://github.com/epiphany-platform/epiphany/issues/3196 - it can help prevent from having mismatched checksums in the future for container images.


Verify backends interchangeability

Directory test/python currently contains (in PR #421) an empty test file named test_simulator_interfaces.py.
Write this test. It will run the various simulators on various inputs, using the backend wrappers in qiskit/backends/local.
Then it will verify that the outputs are similar (distinguish between regular and state vector simulators, in terms of inputs validity and expected outputs; expect identical outputs across state vector simulators, and find a way to compare output of regular simulators containing measurements).

This was done in #435, where several cross-simulator tests were added. But feel free to add more later. I will close this as resolved.


Doppins automated PRs

Is there any chance a collaborator could disable doppins and cleanup all of the conflicting PRs spanning back to 2017..?  That would eliminate 38 unnecessary/wont fix PRs from the total 130 currently open.
The doppins integration seems to have been a problem since it was enabled... and dependency management in general.
Refs:
#1312
#1678
CC @mcrowson @jneves

We're still discussing internally pinning to specific requirement numbers or not. That said, Doppins hasn't really been used to stay up to date with pinning anyhow. Doppins is now off and the old PRs removed. Might go back to it in the future, but currently the PRs are just unused baggage.

Awesome, thanks!


Added Contribution guidelines.

Added CONTRIBUTING.md containing contribution guidelines.

Merging it myself as there isn't anything much to be modified here. Any modifications can be done later.


Broadcast cache updates should handle cases where the window hasn't loaded script yet

Library Affected:
workbox-broadcast-cache-updates
In navigation requests where the service worker responds from the cache, its very likely that any BroadcastChannel message won't be received by the window because it'll be sent before the window has finished loading the page (and the page's script).
We could work around this issue by storing pending cache updates in IDB if we detect that clients.matchAll() doesn't include any clients where type equals window.
Unfortunately it's not clear when the best time to retry broadcasting those messages is, since it will most likely require communication from the window.
We've discussed the idea of a workbox module that runs on the window, and I think something that informs the Service Worker when it's ready to receive messages would be a great addition.

The nuance here is that "the window" is a misnomer. A single service worker instance can control multiple window clients—e.g. two different tabs with the same page loaded in each of them.
So, there are edge cases where one window client is "ready" and another client isn't, but the client that's "ready" is buried under a pile of other open tabs, while the client that the user is actually looking at isn't ready (yet).

The Client API does expose the window's visibilityState, so we could potentially not send notifications if no clients are visible. That being said, I think it makes more sense to send them anyway and let apps decide how/when to dismiss them.
The main problem, though, occurs with normal usage of an app where the navigation request returns stale from the cache and the broadcast update sends the message before the requesting window is ready to receive it.
For navigation requests, we could solve this by only broadcasting the message if a client window exists matching the fetch event's clientId. This would avoid the issue of false positives from other tabs being open.


Add ability to get by platform

In cases where you need to inspect what the current version of the iOS or Android app is at, it would be nice if the utility allow one to get the version by specified platform: get-version --platform ios.

This makes sense if we have the ability to set per platform versions like in #1. Happy to integrate.


upgrade flask to 2.2.5

API change check
API changes are not detected in this pull request.

/check-enforcer override

/check-enforcer reset


[FormBuilder] What to do with formConfig then building the form dynamically ?

The documentation explain this scenario:
let (formState, formConfig) =
    Form<Msg>
        .Create(OnFormMsg)
        .AddField(
            BasicInput
                .Create("firstname")
                .WithDefaultView()
        )
        .Build()

So caching formConfig is possible directly.
But if you have something like:
let createForm (userInfo : UserInfo) =
    Form<Msg>
        .Create(OnFormMsg)
        .AddField(
            BasicInput
                .Create("firstname")
                .WithValue(userInfo.Firstname)
                .WithDefaultView()
        )
        .Build()

Then you formConfig isn't cached by default. Current solution is to use a mutable variable to store the formConfig:
let mutable formConfig = Unchecked.defaultof<Types.Config<Msg>>

// Then to init the form
            let (state, config) =
                createForm currentUser userInfo

            formConfig <- config

            let (formState, formCmd) =
                Form.init formConfig state

Which isn't perfect. We should make formConfig cached by default and probably handle by the library in a special spaces.
One thing to take care is to destroy an old formConfig that is not useful anymore.

I moved this issue to another repository, further work will take place there.


ProtoStream will throw an Exception if a remote-task is executed in T…

https://issues.redhat.com/browse/ISPN-14131

checked with upstream, works as expected

Merged, thanks

This brings up the question, should we decided to serialize/marshall immutable collections in protostream? We did with jboss marshalling, which may end up causing issues for people migrating.

This brings up the question, should we decided to serialize/marshall immutable collections in protostream? We did with jboss marshalling, which may end up causing issues for people migrating.

It's possible, but we would need an alternative mechanism in ProtoStream to AbstractCollectionAdapter interface that immutable collections use.
A workaround for a users with immutable collections is to make it so they return the field via methods and do the conversion themselves, e.g.:
class Example {
   List<String> unmodifiableEntries;

   @ProtoFactory
   public Example(List<String> entries) {
       this.unmodifiableEntries = Collections.unmodifiableList(entries);
   }
   @ProtoField(number = 1, collectionImplementation = ArrayList.class)
   List<String> entries() {
      return new ArrayList<>(unmodifiableEntries);
   }
}


add new 'jira create' command

Creates a new, simple Jira ticket using a set of configuration values and a given ticket summary/title.
$ osdctl jira create "Fix a thing"
Successfully created ticket:
https://issues.redhat.com/browse/OSD-12345

Resolves https://issues.redhat.com/browse/OSD-18777

@typeid this is a good point, and definitely brings up the concern of osdctl expanding too much.
A lot of times, on-call SREs (or after shift) find themselves doing unplanned work. This could be updating an operator, fixing an SOP, or otherwise. The intent of this ticket was to allow the SRE to quickly make a Jira ticket to track that effort, all without leaving their terminal and having to go through Jira's web UI.
It's possible the title and description of this new function do a poor job selling that intent - the current name does indeed make it seem like this is just for generically creating any old Jira ticket. I'm open to any suggested changes, or if people feel that this doesn't belong in osdctl at all.

/hold
Important discussion going on right now that I think should be resolved before we move forward

/unhold
We've decided to move forward with this change for now, and reevaluate the purpose and structure of osdctl going forward.

/lgtm


Sensor values from Essent integration are mixed up when restarting Home Assistant

The problem

I'm using the Essent integration to retrieve my gas and electricity meters and tariffs. Multiple meters are returned:

sensor.essent_elektriciteit_l
sensor.essent_elektriciteit_l_2
sensor.essent_elektriciteit_n
sensor.essent_elektriciteit_n_2
sensor.essent_gas_n
sensor.essent_gas_n_2

When restarting Home Assistant the sensors are sometimes mixed up. The same sensor is not always appended with _2. This results in my sensor values being mixed up. It appears the appended numbers are assigned randomly on start.
Environment

Home Assistant release with the issue: 0.104.3
Last working Home Assistant release (if known): -
Operating environment (Hass.io/Docker/Windows/etc.): Hass.io
Integration causing this issue: essent
Link to integration documentation on our website: https://www.home-assistant.io/integrations/essent/

Problem-relevant configuration.yaml

- sensor:
  - platform: essent
    username: !secret essent_username
    password: !secret essent_password

Traceback/Error logs

Additional information

@TheLastProject you could use the EAN to create a unique_id for every single sensor.

Adding the unique_id fixes the issue! I've created a pullrequest for the fix.


No example of optionally associating to subnet

There are no listed variables or options to associating the NSG to an existing subnet - Please provide in documentation.

Hi @Lucero7919 , thanks for opening this issue. There are examples in module vnet showing how to associating nsg to subnet. Please refer https://github.com/Azure/terraform-azurerm-vnet#example-adding-a-network-security-rule-for-ssh.
Sorry that I have to close this issue temporarily. Please reopen it when anything unclear.


Why does AuthServer depend on AdministrationEntityFrameworkCoreModule and SaaSEntityFrameworkCoreModule?

As title.

@nodew This is to know about the permission and features.

@antosubash Thanks for the confirmation. But I don't think it's a good practice in microservice architecture.


⚠ Bofrost.fr has degraded performance

In 05d3bef, Bofrost.fr (https://www.bofrost.fr) experienced degraded performance:

HTTP code: 200
Response time: 1332 ms

Resolved: Bofrost.fr performance has improved in dc74639 after 11 minutes.


Using PyGLM and PyOpenGL together

I try to use PyGLM and PyOpenGL together but it appears there is no way to glue these two libs. Maybe I'm missing something.

value_ptr helps.


Bug | wrong word in text

Describe the Bug
In this page, at the bottom there is this line:
Next, you will orchestrate de first and main account of the project, the management account.

It should be changed to:
Next, you will orchestrate the first and main account of the project, the management account.

this is the current page with the typo
https://leverage.binbash.co/try-leverage/leverage-project-setup/#:~:text=Next%2C you will orchestrate de first and main account of the project%2C the management account.

Solved with https://github.com/binbashar/le-ref-architecture-doc/pull/197#event-10798580902


Segfaults in 1.7.7 with TSI enabled

(Log attached at bottom)
Environment info:

InfluxDB OSS 1.7.7
TSI enabled

Config:
bind-address = ":8088"
reporting-disabled = false

[admin]
bind-address = ":8083"
enabled = true
https-certificate = "/etc/ssl/influxdb.pem"
https-enabled = false

[[collectd]]
enabled = false

[continuous_queries]
enabled = true
log-enabled = true
run-interval = "1s"

[coordinator]
log-queries-after = "5s"
max-concurrent-queries = 0
max-select-buckets = 0
max-select-point = 0
max-select-series = 0
query-timeout = "0s"
write-timeout = "10s"

[data]
cache-max-memory-size = "8g"
cache-snapshot-memory-size = 26214400
cache-snapshot-write-cold-duration = "10m"
compact-full-write-cold-duration = "4h"
dir = "/var/lib/influxdb/data"
engine = "tsm1"
index-version = "tsi1"
max-concurrent-compactions = 0
max-points-per-block = 0
max-series-per-database = 0
query-log-enabled = true
trace-logging-enabled = false
wal-dir = "/var/lib/influxdb/wal"
wal-logging-enabled = true

[[graphite]]
enabled = false

[http]
auth-enabled = false
bind-address = ":8086"
enabled = true
https-certificate = "/etc/ssl/influxdb.pem"
https-enabled = false
https-private-key = ""
log-enabled = false
max-connection-limit = 0
max-row-limt = 10000
pprof-enabled = false
realm = "InfluDB"
shared-secret = ""
write-tracing = false

[meta]
dir = "/var/lib/influxdb/meta"
logging-enabled = true
retention-autocreate = true

[monitor]
store-database = "_internal"
store-enabled = true
store-interval = "10s"

[[opentsdb]]
enabled = false

[retention]
check-interval = "30m0s"
enabled = true

[shard-precreation]
advance-period = "30m0s"
check-interval = "10m0s"
enabled = true

[subscriber]
enabled = true
http-timeout = "30s"

[[udp]]
enabled = false

Logs:
influxdb_segfault_out.log

They moved back to in-mem and are not experiencing this panic anymore but it would still be good to know what caused this in TSI.  Given their use case, I recommend they stay on TSI for future workloads.

Roblox (the prospective customer who experienced this issue) has a restart time of multiple hours now that they're using in-mem indexing.  This is not only unacceptable for them in production but making troubleshooting extremely difficult.  The goal is still get them successfully running TSI.

This will be difficult to reproduce. The panic originates from here:
https://github.com/influxdata/roaring/blob/fc520f41fab6dcece280e8d4853d87a09a67f9e0/bitmapcontainer.go#L910
So element k doesn't exist in arrayContainer.content which is weird because c is just len(container).

@jacobmarble trace it back to InfluxDB and it should be possible to audit the code to understand where the race is, and if we already fixed it.

Update from prospect per Jacob:
We have been experiencing the issue from within 24 hours or starting the new instances of influxdb. That would put the total data size at around 200GB or less when this started.

Shard duration is 6h and we have seen this within 24 hours of starting a 4w retention policy (which is then changed to 2w).

I used influx_inspect report -detailed on shards from 7/23 to get these values (I didn't want to run this against shards that might actually be used). The report tells me that between all three shards there are 7,371,644 series.

@jacobmarble , Issue occurred on 1.7.2 as well

From user:
"""
I've started an experiment to add a few more data points and narrow down the problem I haven't a been running it too long, but I have some preliminary results that might help with the debugging.
Hypothesis: The change which is causing the observed segfaults is using the tsi1 index
Process: Keeping one server unchanged as the control, modify one server in each of the following ways then observe for future segfaults:
server 1: change retention length 4w -> 2w
server 2: change shard duration length from 6h -> 12h
server 3: change indexing method tsi1 -> inmem
So far, I have observed that server 1 and 2 are still seg faulting. I have not gathered enough data to confirm is server 2 is still seg faulting on a 6h interval or if the interval has changed (I would expect it to change to 12h if anything), but there are definitely still seg faults occuring. Server 3 has not seg faulted since the setting change (~24h/4 shard changes).
This is still not conclusive. I will let you know once I have enough data to really state anything with strong confidence, but right now it's at least a hint I think.
"""

@samhld I have been investigating this issue today and have a pretty good understanding of the issue. I'm figuring out the most appropriate way to resolve it.

@e-dard great!  I'll keep an eye out!

@e-dard please share your theory on what's happening so that someone else can continue making progress on this.

@jacobmarble I shared it internally on the linked EAR (August 8th update).

This probably isn't an issue in 2.x because of @zeebo 's limit work.
Ask @e-dard for more info about root cause and possible repro.

I'm seeing the same issue with v1.7.9. Can you reopen this?

This will be difficult to reproduce. The panic originates from here:
https://github.com/influxdata/roaring/blob/fc520f41fab6dcece280e8d4853d87a09a67f9e0/bitmapcontainer.go#L910
So element k doesn't exist in arrayContainer.content which is weird because c is just len(container).

From the crash log, it's not an out-of-range panic. The index is valid but the memory itself is not.
Jul 22 13:00:10 chi1-ldb18 influxd[92934]: [signal SIGSEGV: segmentation violation code=0x1 addr=0x7f6ce031daac pc=0x95e125]

Above message says memory address 0x7f6ce031daac is not mapping to the process address space (code=0x1).
Given that the TSI uses mmap and the roaring bitmap container also makes use the mapped data, I'd guess it's a race-condition that the log file gets unmap'd while the query is still using the mmap'd slice .
The log does show that there was TSI compaction at the time of crash.
I've not figured out the exact scenario to reproduce this bug anyway.
Jul 22 13:00:10 chi1-ldb18 influxd[92934]: ts=2019-07-22T18:00:10.255627Z lvl=info msg="Removing index file" log_id=0GnOmxlG000 index=tsi tsi1_partition=6 trace_id=0GniIOyl000 op_name=tsi1_compact_to_level tsi1_level=2 path=/var/lib/influxdb/data/PerfmonWeb/4w/93/index/5/L1-00000001.tsi
Jul 22 13:00:10 chi1-ldb18 influxd[92934]: ts=2019-07-22T18:00:10.256615Z lvl=info msg="TSI level compaction (end)" log_id=0GnOmxlG000 index=tsi tsi1_partition=6 trace_id=0GniIOyl000 op_name=tsi1_compact_to_level tsi1_level=2 op_event=end op_elapsed=3096.944ms
Jul 22 13:00:10 chi1-ldb18 influxd[92934]: ts=2019-07-22T18:00:10.342589Z lvl=info msg="TSI log compaction (start)" log_id=0GnOmxlG000 index=tsi tsi1_partition=1 trace_id=0GniIaPW000 op_name=tsi1_compact_log_file tsi1_log_file_id=3 op_event=start
Jul 22 13:00:10 chi1-ldb18 influxd[92934]: unexpected fault address 0x7f6ce031daac
Jul 22 13:00:10 chi1-ldb18 influxd[92934]: fatal error: fault
Jul 22 13:00:10 chi1-ldb18 influxd[92934]: [signal SIGSEGV: segmentation violation code=0x1 addr=0x7f6ce031daac pc=0x95e125]
Jul 22 13:00:10 chi1-ldb18 influxd[92934]: goroutine 126724

This is unrelated to EAR 1315, please ignore it.

Ben could you put this on your list after malloc and OOB backup errors.  Ideally to start no later than w/c April 20, but sooner if it works.

This issue also seems to be caused by closing iterators too soon. A fix has been merged in: https://github.com/influxdata/influxdb/pull/18203

Is this one fixed now? Can we close it?

No movement for two weeks, assuming that this is all done with now.  I'm unassigning  engineering, but give me a shout if you need more help.

@8none1 I'm not who you're relying on to know if this is closable.   As I understood it, this was a bug that edge cases came across.  If it is not, the prospect that brought this up is no longer interested in InfluxDB and this is okay to close.
If it is, it should be fixed right?  Or has it been?

Yeah, the seg fault is fixed.  I think that's probably a wrap for this issue then.

Hi,
I'm experiencing this same problem, both in version 1.7.7 and 1.8.6.
Is there a version prior to 2.0 that fixes this problem?

Sep 28 22:48:03 us-infra-influx-2 influxd[23109]: unexpected fault address 0x7f6fe40fab01 Sep 28 22:48:03 us-infra-influx-2 influxd[23109]: fatal error: fault Sep 28 22:48:03 us-infra-influx-2 influxd[23109]: [signal SIGSEGV: segmentation violation code=0x1 addr=0x7f6fe40fab01 pc=0x9d3f7e] Sep 28 22:48:03 us-infra-influx-2 influxd[23109]: goroutine<PHONE_NUMBER> [running]: Sep 28 22:48:03 us-infra-influx-2 influxd[23109]: runtime.throw(0x151e28d, 0x5) Sep 28 22:48:03 us-infra-influx-2 influxd[23109]: #011/usr/local/go/src/runtime/panic.go:617 +0x72 fp=0xc00c73da70 sp=0xc00c73da40 pc=0x42f482 Sep 28 22:48:03 us-infra-influx-2 influxd[23109]: runtime.sigpanic()
We are still getting Segmentation issues with Version 1.7.9. The influxd service restarted soon after the segmentation violation which started causing write failures. Has the issue resurfaced again ?

Mar 28 06:52:09 influx influxd-systemd-start.sh[14374]: #011/usr/local/go/src/net/http/server.go:2928 +0x384
Mar 28 06:52:10 influx systemd[1]: influxdbsud.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
Mar 28 06:52:10 influx systemd[1]: influxdbsud.service: Failed with result ‘exit-code’.
Mar 28 06:52:10 influx systemd[1]: influxdbsud.service: Consumed 1month 6d 13h 29min 42.403s CPU time.
Mar 28 06:52:10 influx systemd[1]: influxdbsud.service: Scheduled restart job, restart counter is at 1.
Mar 28 06:52:10 influx systemd[1]: Stopped InfluxDB is an open-source, distributed, time series database.
Mar 28 06:52:10 influx systemd[1]: influxdbsud.service: Consumed 1month 6d 13h 29min 42.403s CPU time.
Mar 28 06:52:10 influx systemd[1]: Starting InfluxDB is an open-source, distributed, time series database…
Mar 28 06:52:10 influx influxd-systemd-start.sh[116443]:
Mar 28 06:52:10 influx influxd-systemd-start.sh[116440]: InfluxDB started
Mar 28 06:52:10 influx systemd[1]: Started InfluxDB is an open-source, distributed, time series database.
Mar 28 06:52:10 influx influxd-systemd-start.sh[116441]: ts=2024-03-28T06:52:10.842072Z lvl=info msg=“InfluxDB starting” log_id=0oCy7trW000 version=1.8.10 branch=1.8 commit=688e697c51fd
System Environment:
Operating System: Ubuntu 22.04 on Azure
InfluxDB Version: 1.8.10
CPU: 32vCPU
RAM: 128GB
Database Size: 80GB
Disk Type: SSD
cache-max-memory: 32GB
Index Type: TSM
Any help/suggestions are appreciated in advance.


Adding GitHub specific instructions, to ease the workflow of people like me.

I'd move this after the installation section and before Licence into a new section Usage with GitHub.
There's nothing special about github specifically that warrants mentioning these instructions early in the README.

Right.
Nothing special about github to put it here.
I just want it "somewhere".

Do tell me if the change of position of these instructions suit you.


Implement support for API Gateway v1 and v2

Description
Fixes #349.

TODOs
Read the Gruntwork contribution guidelines.

[x] Update the docs.
[x] Run the relevant tests successfully, including pre-commit checks.
[x] Ensure any 3rd party code adheres with our license policy or delete this line if its not applicable.
[x] Include release notes. If this PR is backward incompatible, include a migration guide.

Release Notes (draft)

Added / Removed / Updated [X].
Implement support for inspecting and nuking API Gateways (v1 and v2).
Migration Guide

LGTM, except for one thing, I think it would be good to have one or two mocks for what a config yaml file would look like with API Gateway, and a test for that mock.

Thanks for review! I've introduced complete mock tests for configuration with APIGateway v1 and v2 in 695ac8f

Thanks for reviews! Going to merge this in now.
I'll release it as backward incompatible given https://github.com/gruntwork-io/cloud-nuke#choosing-a-new-release-tag


Can't deploy Flux standalone to GKE on Kubernetes 1.6.2

I'm trying to deploy Flux standalone to my Kubernetes cluster running in Google Cloud (with GKE) and during the deployment it gives me the following error:
serviceaccount "weave-flux" created
deployment "weave-flux-agent" created
clusterrolebinding "weave-flux" created
Error from server (Forbidden): clusterroles.rbac.authorization.k8s.io "weave-flux" is forbidden: attempt to grant extra privileges: [{[*] [*] [*] [] []} {[*] [] [] [] [*]}]<EMAIL_ADDRESS>[system:authenticated] map[]} ownerrules=[{[create] [authorization.k8s.io] [selfsubjectaccessreviews] [] []} {[get] [] [] [] [/api /api/* /apis /apis/* /healthz /swaggerapi /swaggerapi/* /version]}] ruleResolutionErrors=[]

I replaced my corporate email from the output to<EMAIL_ADDRESS>for privacy/security reasons.
Steps to reproduce

Create a Kubernetes cluster in Google Container Engine. Make sure that the Kubernetes version is 1.6.2. I did it with this Terraform script:
resource "google_container_cluster" "gke" {
  name               = "my-gke"
  zone               = "europe-west1-b"
  initial_node_count = 1
  node_version       = "1.6.2"

  additional_zones = []

  master_auth {
    username = "foo"
    password = "var"
  }

  node_config {
    oauth_scopes = [
      "https://www.googleapis.com/auth/compute",
      "https://www.googleapis.com/auth/devstorage.read_only",
      "https://www.googleapis.com/auth/logging.write",
      "https://www.googleapis.com/auth/monitoring",
    ]
  }
}

Do the gcloud step to get a hold of the kubectl configuration and then:
$ kubectl apply -n kube-system -f "https://cloud.weave.works/k8s/v1.6/flux"

You should see the same output stated above.

Thanks Carlos. Clearly the GKE environment is a more locked down than your average Kubernetes cluster! We'll have to figure out what permissions we are allowed to grant. Do you know GKE well?

@squaremo unfortunately I do not have that much experience with GKE, sorry. Is there anything else I can help you with to debug this issue?

The note under here https://cloud.google.com/container-engine/docs/role-based-access-control#defining_permissions_in_a_role suggests that you need to assign your own account a role with the permissions you wish to grant. If you have not already, it may be worth trying that.

@squaremo got it!
I just double checked and I'm the owner of the project. I also created a new Service Account in Google Cloud to ensure that it also has the right role (I also made the key an owner of the project).
Same story.

I just double checked and I'm the owner of the project. I also created a new Service Account in Google Cloud to ensure that it also has the right role (I also made the key an owner of the project).
Same story.

The note in the GKE docs suggests that you need to give your account a cluster-admin role, and I am not clear from your description that's what you did. In particular, the docs imply it's something done through kubectl, rather than through the GKE user interface. (Admittedly I am not familiar with the latter)

@mongrelion by default your user won't be assigned the cluster-admin role. You can assign the role by running kubectl create clusterrolebinding john-cluster-admin --clusterrole=cluster-admin<EMAIL_ADDRESS>


[BUG] 2.9 GH workflows are failing with Unresolved reference: CAT_INDICES

What is the bug?
WFs on the 2.9 branch are failing with
e: /home/runner/work/alerting/alerting/alerting/src/main/kotlin/org/opensearch/alerting/settings/SupportedClusterMetricsSettings.kt: (90, 35): Unresolved reference: CAT_INDICES
e: /home/runner/work/alerting/alerting/alerting/src/main/kotlin/org/opensearch/alerting/settings/SupportedClusterMetricsSettings.kt: (97, 35): Unresolved reference: CAT_SHARDS
e: /home/runner/work/alerting/alerting/alerting/src/main/kotlin/org/opensearch/alerting/util/clusterMetricsMonitorHelpers/SupportedClusterMetricsSettingsExtensions.kt: ([49](https://github.com/opensearch-project/alerting/actions/runs/8240038860/job/22534582093#step:5:50), 47): Unresolved reference: CAT_INDICES

e: /home/runner/work/alerting/alerting/alerting/src/main/kotlin/org/opensearch/alerting/util/clusterMetricsMonitorHelpers/SupportedClusterMetricsSettingsExtensions.kt: (66, 47): Unresolved reference: CAT_SHARDS

Example: https://github.com/opensearch-project/alerting/actions/runs/8240038860/job/22534582093
How can one reproduce the bug?
Steps to reproduce the behavior:

Go to '...'
Click on '....'
Scroll down to '....'
See error

What is the expected behavior?
A clear and concise description of what you expected to happen.
What is your host/environment?

OS: [e.g. iOS]
Version [e.g. 22]
Plugins

Do you have any screenshots?
If applicable, add screenshots to help explain your problem.
Do you have any additional context?
Add any other context about the problem.

This is resolved, closing


when using http to communicate with a separate funktion container lets configure the http endpoint to bridge

when using http to communicate with a separate funktion container lets configure the http endpoint to bridge

[merge]


Fix demo link on README

Signed-off-by: fermayo<EMAIL_ADDRESS>

Ah, thanks, I changed my GitHub username.


No error handling for MQTT connection problems

When connection to the MQTT broker fails in a permanent way (SSL, connection refused, etc) there is no proper error handling.  It just dumps a standard python stack trace.

Correction, SSL failures go by silently and retry forever.

Implemented in local repo.  Need to test followings scenarios before committing:

Initial connect fails due to Connection Refused
Initial connection fails due to Connection Reset
Failure to reconnect after 60 seconds

Added in 1e843f44eb5975c8ff181b94a411a31327dda6eb


Fix EditorConfigParser

PR for #48

@bennyn Fixed. Could you confirm and merge it?
Thanks.

Great that you wrote a fix! I build the plugin from your patch-for-issue48 but when I am opening an .editorconfig file now, then I don't have any syntay highlighting for it. :( Also the Navigator does not seem to work anymore. Could you use the plugin successfully? @Yserz: Does the EditorConfig Editor work for you?

It works in my environment.

It's working fine here too.
Product Version: NetBeans IDE 8.0.1 (Build<PHONE_NUMBER>40)

Ok! Thanks for testing! Then I will merge it. You both are running Mac OS, right?


feat(Dashboard): Redesigns sidebar with category labels

Adds category labels to sidebar
Adds localization support to sidebar and page titles

closes #289

Thoughts on having an "Overview" or "General" category for the dashboard section?

Looks great! Thoughts on having an "Overview" or "General" category for the dashboard section?

Good idea! Added it for when we eventually add in #251 as it needs a home.


Automated Package Upgrade

:tada: This PR is included in version 7.7.132 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Code First: Implement inheritance support

[x] APIs for configuring base type
[ ] Base type discovery convention
[ ] Support for ignoring discovered base type
[ ] Support for ignoring inherited properties
[x] APIs for relational discriminator annotations
[x] Discriminator introduction convention (readonly by default)
[x] Relational model validation rule to ensure correct inheritance mapping

@AndriySvyryd Two questions:

Will Code First take care uniquifying/collapsing properties in the hierarchy? For example, what would happen to the Name properties:
 class Person
 {
 }

 class Student : Person
 {
     public string Name { get; set; }
 }

 class Teacher : Person
 {
     public string Name { get; set; }
 }

Do you also intend to also update CSharpModelGenerator to serialize inheritance metadata in model snapshots?

@bricelam

Depends. When using a ModelBuilder with just the core conventions the EntityTypes for Student and Teacher each will have a Name property, while the one for Person will have none.
When also using the relational convention the TPH convention would create a discriminator shadow property and uniquify all the column names that would end up mapped to Persons table. In this case it will result in columns Name and Name1 named deterministically.
No, that is not part of this issue.

Sounds good. Thanks.

Discriminator API:
// If it's a string we just keep using type name for values
modelBuilder.Entity<Plant>().Discriminator<string>("Type_Of_Entity");
modelBuilder.Entity<Plant>().Discriminator(p => p.TypeOfEntity);

modelBuilder.Entity<Rose>().Discriminator(p => p.Genus).Value("Rose");
modelBuilder.Entity<Daisy>().Discriminator(p => p.Genus).Value("Daisy");

modelBuilder.Entity<Plant>().Discriminator(p => p.Genus);
modelBuilder.Entity<Rose>().Discriminator().Value("Rose");
modelBuilder.Entity<Daisy>().Discriminator().Value("Daisy");

// plus support for typeof() overloads
modelBuilder.Entity<Plant>().Discriminator(p => p.Genus)
    .Value<Rose>(PlantGenus.Rose)
    .Value<Daisy>(PlantGenus.Daisy);

EF6 had the ability to use multiple discriminator properties to avoid the use of a dedicated discriminator column. You can set that it up so that if a subclass property is not null, that tells EF that the correct type is the subclass that owns the property. Is this being tracked anywhere?


Include user/packager provided CFLAGS, CPPFLAGS, and LDFLAGS

Hello,
We use this patch in the Debian package for rhdf5filters and thought it would be nice to share.
Thanks,

Thanks for this. My understanding is that's generally better practice to make sure the C code is being compiled with the same options R itself was built with, so I've tried to fix the issues that way in 9ae6a8d57aec39a778a6beaf642bdfd94c39025b and
537a7ec.
Hopefully that works for you without the patch going forwards.  Please let me know if that's not the case.

@grimbough I'll test it after the next release, thanks!


Consider Enabling #![deny(missing_docs)] Attribute

Since this is a library with the intention of maintaining a public API, it seems like a good idea to enable the #![deny(missing_docs)] attribute at some point to ensure that all exposed items (fields, structs, etc.) have proper documentation attached.
Enabling this right now is probably not a good idea since the s3 module is missing documentation for a lot of its exposed features. This should be kept on-hold until s3 is generated like the rest of the code.
Alternatively, this could be enabled progressively by adding the attribute to individual modules over time, then finally moving it to be a global attribute once it's ready.

This is a good starting point for someone interested in improving our codegen.  First pass could be adding the flag to one of the types of codegen we have, such as ec2 or rest-json.

This is easier to implement now.  We should knock this out.

I'm not sure this is feasible now that service code isn't generated at build time. We're kind of at the mercy of botocore's definitions here, and it would suck to not be able to build anymore if botocore is missing documentation for a shape/member/operation. I do think that this would be nice to have on rusoto_core and rusoto_credential, though.


Remove issue "natural=cape should be a standalone point based on its tags"

The wiki for natural=cape says that it should be a node as a part of the coastline. So this issue is in direct conflict with the wiki.
https://wiki.openstreetmap.org/wiki/Tag:natural%3Dcape#How_to_map

Looks like this wording was just added a year ago in this change
The talk page and history suggest that some disagreement going on about how to use this tag.
We can't really enforce the rule that a cape has to be a vertex along a coastline.  Can we leave it as-is?  It seems a lot more intuitive to me to let people just drop a point where they want the label for the cape to be.

I think the best solution is to just add vertex as a valid geometry for capes and let people map as they see fit.

I think the best solution is to just add vertex as a valid geometry for capes and let people map as they see fit.

Good suggestion 👍   this is what I did..


Allow access to dev files

If not used carefully I can get my python code on a browser
https://www.youtube.com/watch?v=jv3BH3PW-iE
please consider any help from my side

Please use StackOverflow for questions about how to use the library.


⚠ GitHub has degraded performance

In 2586e39, GitHub (https://www.githubstatus.com/api/v2/status.json) experienced degraded performance:

HTTP code: 200
Response time: 435 ms

Resolved: GitHub performance has improved in ad87f75.


Lab export fix, local dev ?

It seems that the lab/export does only after a while loads the latest content of the repository and in this case it gives an Server Error
https://site.usegalaxy.org.au/lab/export?content_root=https://raw.githubusercontent.com/paulzierep/galaxy_tool_extractor/refs/heads/microgalaxy_lab/communities/microgalaxy/lab/base.yml
@neoformit any idea ?

Hi Paul, please use the Lab Engine server, this is no longer maintained on site.usegalaxy.org.au (the Media Site):
https://labs.usegalaxy.org.au/?content_root=https://raw.githubusercontent.com/paulzierep/galaxy_tool_extractor/refs/heads/microgalaxy_lab/communities/microgalaxy/lab/base.yml
Looks good to me! The server caches these pages, so if you are in development please add &cache=false to the URL:
https://labs.usegalaxy.org.au/?content_root=https://raw.githubusercontent.com/paulzierep/galaxy_tool_extractor/refs/heads/microgalaxy_lab/communities/microgalaxy/lab/base.yml&cache=false

Thanks the cache parameter is really useful ! However we should add to the documentation, that actually github .raw can be the issue as well: https://github.com/orgs/community/discussions/46691 :)

Still, it seems, that sometime especially the sections seem not to render directly. E.g. this
https://raw.githubusercontent.com/paulzierep/galaxy_tool_extractor/refs/heads/microgalaxy_lab/communities/microgalaxy/lab/sections/5_tools.yml is not rendered, even with the cache option: https://labs.usegalaxy.org.au/?content_root=https://raw.githubusercontent.com/paulzierep/galaxy_tool_extractor/refs/heads/microgalaxy_lab/communities/microgalaxy/lab/base.yml&cache=false

OK working now, seems to be some lack between GitHub, GitHub Raw Data and the Engine Server!

Thanks Paul, yes there are multiple layers of caching with github raw content being cached for 300s. It can be problematic but I don't think I've experienced more than a few moment's delay in the past! I have doc'd this but it's not yet deployed on labs.usegalaxy.org.au.

Ok, this is now in the "How do I make one" dropdown on the docs page


Fix flaky test

From the fail logs on issue #871, the flaky tests all happen after a
HNCConfigue update, but the supposed follwing updates could not be
detected. We give these lines more time in Eventually() so that
HNCConfigue update gets more time to propogate. However, I was not able
to reproduce the flaky test results when running locally, so I cannot
confirm that they are fully fixed by this pr.
Tested: make test

/assign @yiqigao217
/assign @adrianludwin

/lgtm
/approve

/approve


Use updated apigw endpoint w/ secure channel

Change is running successfully in internal service tests.

Codecov Report

Merging #373 into master will decrease coverage by <.01%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master     #373      +/-   ##
==========================================
- Coverage   54.48%   54.48%   -0.01%
==========================================
  Files         341      341
  Lines       58434    58434
==========================================
- Hits        31837    31836       -1
- Misses      26597    26598       +1

Impacted Files
Coverage Δ

src/test-apps/TestCASE.cpp
95.89% <0%> (-0.23%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f83b6ce...3306e72. Read the comment docs.


How to get artboard's background color?

I cannot find API in documents to get artboard's background color, and sketch cannot export SVG with artboard's background color.

+1


new model E coli W3110; testing

This let's us upload the 55th model in from Jon's paper. Eventually, this will be superseded by SBRG/BIGG2#68.
removes a bunch of whitespace
EX_glc__D_e is being specified in reaction-hash-prefs.txt, so we test for that

See also SBRG/BIGG2#134

Need to make sure we are testing model dumping

Ready to merge


Fix gfx_hal::memory::Properties to match VkMemoryPropertyFlagBits

This commit swaps the COHERENT and CPU_VISIBLE visible to match the order and values defined by Vulkan.
See https://www.khronos.org/registry/vulkan/specs/1.1-extensions/man/html/VkMemoryPropertyFlagBits.html
PR checklist:

[x] make succeeds (on *nix)
[x] make reftests succeeds
[x] tested examples with the following backends: vulkan
[x] rustfmt run on changed code

That looks good, thank you!
Do you mind also fixing the Vulkan backend to simplify the mapping of this flags type?

I looked into it but I am not familiar enough with the codebase to update the Vulkan backend. I was debugging my own code when I noticed the discrepancy between the values of gfx-hal and the Khronos API, but I am just getting started with this lib.

Ok, thanks!
Bors r+

On Apr 4, 2019, at 12:02, Charles Samborski<EMAIL_ADDRESS>wrote:
I looked into it but I am not familiar enough with the codebase to update the Vulkan backend. I was debugging my own code when I noticed the discrepancy between the values of gfx-hal and the Khronos API, but I am just getting started with this lib.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or mute the thread.


Unhandled signal

For test case sending multiple same signal at the same time; it detects unhandled signal
			env.SignalWorkflow("signal-receive-with-timeout", &message)
			env.SignalWorkflow("signal-receive-with-timeout-clone", &message)
			env.SignalWorkflow("signal-receive-with-timeout", &message)
			env.SignalWorkflow("signal-receive-with-timeout", &message)

The newer SDK also have selector.HasPending but not sure if that is useful in this scenario
Maybe something like below:
	selector.AddFuture(
		timer,
		func(f workflow.Future) {
			if err := f.Get(ctx, nil); err != nil && temporal.IsCanceledError(err) {
				fmt.Println("WORKFLOW CANCELED!!!!!")
				isCancelled = true
			} else {
				fmt.Println("TIMED OUT!!!")
				hasTimedOut = true
			}
		},
	)
	selector.AddReceive(
		sigCh,
		func(c workflow.ReceiveChannel, more bool) {
                        // This handles multiple signals of same type
			for c.ReceiveAsync(valuePtr) {
				fmt.Println("MORE SIGNALS!!!")
			}
		},
	)

Thanks for filing the issue!
I'm not sure in what scenario we want to process the additional signals, and how we can offer this to customers using this library.
That said, when we used ReceiveWithTimeout in our own projects, we do have a use-case for draining the signal channel before using it to ensure we take the latest.
For example, if we were to run ReceiveWithTimeout in a loop:

Receive signal
Workflow continues progressing
While (2) is happening, more signals are still being received, and collected
When the loop comes back round, (1) is instantly fired

To avoid (4), we run something like:
for {
    hasData := sigCh.ReceiveAsync(nil)
    if !hasData {
        break
    }
}

res := channel.ReceiveWithTimeout(ctx, sigCh, &result, timeout)

If this is a common requirement, we can easily back this in.


Select pod resources from binder UI

In the last few weeks, a number of folks have asked for specific resources for specific binders. Examples of these requests are:

can I have a pod with a GPU in it?
can I have a pod with extra memory or CPU?

Kubespawner currently has this functionality through the use of a profile_list.
I'm curious if this is in scope for BinderHub and, if so, what the API would be. I'm imagining a dropdown in the UI that provides the equivalent of the Kubespawner profile list.

There have been previous discussion on this and one of the questions that keeps coming up is: should the launcher or the creator of a repo control the resource request? What should happen if the request can't be satisfied ("recommended resources" vs "required resources")?
From a technical point of view having the repository (the creator) specify it is tricky because it means BinderHub needs to clone the repository in order to find out.
Part of the reason for #712 is that I want to be able to specify resource requests as part of the launch link.
I think it would be great if you could drive this discussion forwards so that it gets to a point where we have a concrete suggestion to implement that does what people would like it to do.

Yes, I was also thinking about this problem in these terms. Not to make this more complicated but I think there are three options here.

Creator of binder repository prescribes resources. This would look something like the singleuser section in the jupyterhub chart:

  singleuser:
    # start jupyter notebook
    cmd: jupyter-lab
    cloudMetadata:
      enabled: true
    cpu:
      limit: 4
      guarantee: 1
    memory:
      limit: 26G
      guarantee: 4G
    nodeSelector:
      cloud.google.com/gke-nodepool: user-pool

Of course, this is more complicated than most users would like to see but it would allow for significant customization.
2. User selects from a set of preordained resource packages. This would look like the functionality that kubespawner profile_list provides. You might imagine allowing something like what I have below in the binderhub chart:
  {
      'display_name': 'Base Env',
      'default': True,
      'spawner_override': {
          'cpu_limit': 1,
          'mem_limit': '512M',
      }
      'display_name': 'Big Memory Env',
      'spawner_override': {
          'cpu_limit': 48,
          'mem_limit': '96G',
      }
  }, {
      'display_name': 'GPU 2  Env',
      'spawner_override': {
          'cpu_limit': 48,
          'mem_limit': '96G',
          'extra_resource_guarantees': {"nvidia.com/gpu": "2"},
      }
  }

Finally, a more flexible REST-like API could expose these options to the user and the Binder UI could provide options for a bunch of options. This follows #712.

My personal thought, without having gone through the step of truly scoping the work, is that (2) is going to be fairly easy and accomplish most of what we will ever need. I agree that (1) will be hard because the binder would need to be inspected before deploying. (3) also seems like a reasonable option but may expose much more flexibility than is really needed.

Is there an update on this? Both 1. and 2. would be useful, but 2. would be enough for what we need.

@rochaporto - I don't think anyone has immediate plans to work on this. Without speaking for the binder team, I assume contributions would be welcome.

We have resources to work on this now. If noone has picked it up we'll give it a go. If there's ongoing work already please let us know.

@betatim can we go ahead and pick this up?

This would be very useful

Can someone please update the status of this issue?


State what screen readers are tested

🚀 Feature request
Motivation
Similar to Reach UI, if the project declares what screen readers and browser combinations are tested, it gives adopters confidence that the components take into account different screen reader quirks to deliver the best UX.
Example
Quote from Reach UI:

Each component is tested with Safari + Voiceover, Firefox + NVDA, and Edge + JAWS. As the project matures we’ll get it audited by Web AIM to ensure that if you pick Reach UI, your app has a solid, accessible foundation.

Bump

Bump

I'm happy to help with this one 👍🏻
As far as I saw components on this library are tested through RTL ( Node ) simulating keyboard interactions there ... ⌨ , and I feel that's probably enough?
For what I can see Reach UI does the same and couldn't find where they state that they test manually with VoiceOver etc... nor any documentation around that process 🤔
One thing we could improve in Reakit's tests is to add jest-axe to them, what do you think @diegohaz ?

jest-axe would be a nice addition! 👍
I manually test components on Safari with VoiceOver and on Windows Firefox/Chrome with NVDA and JAWS. And this represents about 74% of usage according to the most recent survey from WebAIM: https://webaim.org/projects/screenreadersurvey8/#browsercombos
IE 11 is tested and it works with polyfills (that are added automatically by babel). But I usually don't test JAWS with Internet Explorer (testing on IE alone is enough of a nightmare 😅), but I expect it works.

Coolios, I see two pieces of work from this?
Task 1

I manually test components on Safari with VoiceOver and on Windows Firefox/Chrome with NVDA and JAWS. And this represents about 74% of usage according to the most recent survey from WebAIM: https://webaim.org/projects/screenreadersurvey8/#browsercombos

IE 11 is tested and it works with core-js polyfills (that are added automatically by babel). But I usually don't test JAWS with Internet Explorer (testing on IE alone is enough of a nightmare 😅),

Explain this in the documentation, maybe in the homepage at the bottom or in the troubleshooting page?
Task 2
Open a PR adding axe verifications for each component, in the lines of...
import { axe } from 'jest-axe'

test("has no a11 violations", async () => {
  const { container } = render(<Button>button</Button>)
  const results = await axe(container)
  expect(results).toHaveNoViolations()
})

@lluia It probably makes sense to also run the examples through jest-axe. Related to #626 and #624

Thanks for tackling this y'all! Great to hear you've already been testing in different ATs (assistive technologies, like screen readers) @diegohaz. You might consider adding iOS VoiceOver + iOS Safari to that list, as it's a different codebase than MacOS VoiceOver with different bugs.
@lluia here's where Reach UI mentions it: https://reacttraining.com/reach-ui/

Each component is tested with Safari + VoiceOver, Firefox + NVDA, and Edge + JAWS. As the project matures we’ll get it audited by WebAIM to ensure that if you pick Reach UI, your app has a solid, accessible foundation.

I agree more detail would be welcome, but documenting exactly how you're manually testing and the AT/browser versions you last tested with can get tedious.
At the least, stating AT/browser support sends the signal that you're willing to fix bugs people find, which is great.

You might consider adding iOS VoiceOver + iOS Safari to that list, as it's a different codebase than MacOS VoiceOver with different bugs.

Ah yeah! Forgot to mention that one. I do test on VoiceOver on iOS.

@diegohaz should we add that text in the homepage then? 🧐

Bump.
Has anything changed since 2020 in this regard, and does it make sense for an external contributor to send a PR outlining things what @diegohaz stated previously in this thread in project readme and contribution guidelines?


Generic list

See Issue #452
Straight forward to use. Tests with no memory leaks.

I moved this code into its own branch. So I'm going to close this PR and re-issue it from dev-list.


Disable "add site" functionality when using Nova API

The add site functionality is no longer required in the iOS app and is not supported by the new Nova API, so this functionality should be neatly disabled but ideally be fairly simple to bring back to life in future if desired.

It can just be killed with fire, along with all the Airtable logic. I dont want to complicate matters by trying to keep things around but hidden behind toggles, we need to just trim the fat here I think. I'll be carrying around an iPad at all times in case I need to get into the Nova backend so iPhone worries are less.

@philsturgeon I've prepared the linked PR to remove add site and all of the redundant Airtable code. This is the quite the clear-out, and will yield 0.10.3 which I suggest we focus our testing efforts on for the time being.


bta_jv_int.h in wrong folder. (IDFGH-1774)

File in wrong folder:    esp-idf/components/bt/bluedroid/bta/jv/bta_jv_int.h
Correct path is:    esp-idf/components/bt/bluedroid/bta/include/bta_jv_int.h

In what commit?
https://github.com/espressif/esp-idf/tree/master/components/bt/host/bluedroid/bta/jv/include

Sorry my mistake, I was accidetially using old 'local' branch. This issue not present in any git branch.
Close issue please.


Relax verifier check for InitBlockStorageHeader instructions.

Correctly handle secondary substitutions using the existing verification logic for apply instructions. Fixes rdar://problem/26623854.

@swift-ci Please test

@eeckstein Does this look good for Swift 3?

yes, looks good