Add new attribute: battery_last_updated_button

Checklist

[X] I have filled out the template to the best of my ability.
[X] This only contains 1 feature request (if you have multiple feature requests, open one feature request for each feature request).
[X] This issue is not a duplicate feature request of previous feature requests.

Is your feature request related to a problem? Please describe.
Having this info as an attribute of the Battery+ sensor would simplify referencing the button entity to update the associated battery's battery_last_replaced date. Rather than having to resort to templating like:
{{ entity_id | replace("sensor.","button.")|replace("_battery_plus","_battery_replaced") }}
one could use a more elegant and straightforward {{ state_attr(entity_id, 'battery_last_replaced_button') }} template.
Describe the solution you'd like
I tried adding a templated attribute via custom-ui but it is not completely real i.e. It's available in the front-end (custom:battery-state-card) but not the back-end (scripting/dev tools):
state_class: measurement
battery_quantity: 1
battery_type: CR2032
battery_type_and_quantity: CR2032
battery_low: false
battery_low_threshold: 15
battery_last_reported: "2024-02-14T07:01:08.891694"
battery_last_reported_level: "50"
battery_last_replaced: "2024-02-14T07:49:11.536876+00:00"
unit_of_measurement: "%"
device_class: battery
friendly_name: Outside Weather
hide_attributes:
  - templates
templates:
  icon_color: >
    var r = Math.min(255, Math.round(510 - 5.10 * Number(state))); var g =
    Math.min(255, Math.round(5.10 * Number(state))); var h = r * 0x10000 + g *
    0x100; return '#' + ('000000' + h.toString(16)).slice(-6);
  battery_last_replaced_button: >
    return 'button.' + entity.entity_id.split('.')[1].split('_battery_plus')[0]
    + '_battery_replaced';
icon_color: "#ffff00"
battery_last_replaced_button: button.outside_weather_battery_replaced

Describe alternatives you've considered
Too many to mention!
Additional context
homeassistant:

  customize_glob:
    "*.*":
      hide_attributes:
        - templates

    sensor.*_battery:
      templates: &battery_color
        icon_color: >
            var r = Math.min(255, Math.round(510 - 5.10 * Number(state)));
            var g = Math.min(255, Math.round(5.10 * Number(state)));
            var h = r * 0x10000 + g * 0x100;
            return '#' + ('000000' + h.toString(16)).slice(-6);

    sensor.*_battery_level:
      templates: *battery_color

    sensor.*_battery_plus:
      templates:
        <<: *battery_color
        battery_last_replaced_button: >
          return 'button.' + entity.entity_id.split('.')[1].split('_battery_plus')[0] + '_battery_replaced';

Why do you need to access the button when you could make a service call which just requires a device_id that's already supplied in the attributes?
https://andrew-codechimp.github.io/HA-Battery-Notes/services/#battery_notesset_battery_replaced

Um, I was overthinking it? Plus, I was not aware of the service: battery_notes.set_battery_replaced using device_id.
That simplifies my config in custom:auto-entities with custom: battery-state-card to achieve a tap_action to update the battery_replaced_date.  I still have to call a script but the script is simpler now.
alias: Battery Notes - Update Battery Replaced Date
description: Updates the battery last replaced date
fields:
  entity_id:
    name: entity_id
    description: Battery Entity ID
    example: "sensor.outside_weather_battery_plus"
sequence:
  - service: battery_notes.set_battery_replaced
    data:
      device_id: "{{ device_id(entity_id) }}"

type: custom:auto-entities
card:
  type: custom:battery-state-card
  title: Low Batteries
  debug: false
  card_mod:
    class: inline-card
filter:
  include:
    - integration: battery_notes
      domain: sensor
      attributes:
        device_class: battery
      state: < 60
      options:
        name: '{attributes.friendly_name} ({attributes.battery_type_and_quantity})'
        secondary_info: 'Last reported: {attributes.battery_last_replaced_button}'
        tap_action:
          action: call-service
          service: script.battery_notes_battery_replaced
          service_data:
            entity_id: this.entity_id
    - integration: battery_notes
      domain: binary_sensor
      attributes:
        device_class: battery
      state: 'on'
  exclude:
    - integration: mobile_app
sort:
  method: state
  numeric: true

I've requested an enhancement to battery-state-card and @maxwroc marked as a v3.3 milestone:
Modify tap_action + add hold_action & double_tap_action as per Home Assistant Dashboard Actions for tap actions #691.
If he also adds his KString parsing to the entity_id in tap_action then I would be able to reference the new attribute via '{attributes.battery_last_replaced_button}' and avoid the sctipt.

Working out the button entity_id from the battery+ sensor is a lot of messy code hops I'd rather avoid, especially if someone goes and gives it a different entity_id.
Hopefully using the service with the script will suffice?

I think that the tap_action (or even hold_action, and other standard actions) are not a perfect solution for resetting "battery replaced date".
FYI I plan to add features in battery-state-card which will be battery-notes specific. In particular I was thinking about adding a button(s) for standard actions available in battery-notes (e.g. "battery replaced", "changing battery type" etc) So hopefully you won't have to struggle to achieve what you want.
I will allow KString usage in various other places too so you should be able to do it one way or the other

@maxwroc if you need any clarification on how things work in Battery Notes or would like small changes happy to discuss.
There's a discussion forum on this repo you can start a general discussion on if you want and just @ me

  tap_action:     action: call-service     confirmation:       text: Battery Replaced?     service: script.battery_notes_battery_replaced     service_data:       entity_id: this.entity_id

 hold_action:     action: call-service     confirmation:       text: Battery Replaced?     service: script.battery_notes_battery_replaced     service_data:       entity_id: this.entity_id

@danielbrunt57
  tap_action:
    action: call-service
    confirmation:
      text: Battery Replaced?
    service: script.battery_notes_battery_replaced
    service_data:
      entity_id: this.entity_id

@webmogul1 Unfortunately, battery-state-card currently does not recognize confirmation:

Adding device_id as an attribute to battery+ will be very easy since I already do it in the events and I guess will be useful for a lot so I'll add that soon.


Truncate branch or PWD when over character limit

When the git branch or PWD is too long, it can look ridiculous, and even wrap. They need to truncate if they get too long.

had limited time, but got the path truncating. Now just need to do that same with git branches

branches are now truncating. Closing issue.


Despite more times available, dataset read from MultiZarrToZarr returns specific number time steps

This is weird error but here goes...
I am going through the kerchunk steps for ROMS model output files and it works. BUT, no matter how many single file zarr metadata files I pass to MultiZarrToZarr and the resultant translated product, the dataset I read in stops at about a year's worth of time steps. Is there a built in limit somewhere that I need to adjust? These are local files.
The basic code is below. If json_list includes the single file metadata from

1999 and 2000, the Dataset has times from 1999-01-01 to 2000-01-02 instead of 1999-01-01 through 2000
only 1999 the Dataset has times from 1999-01-01 to 2000-01-01
only 2000 the Dataset has times from 2000-01-01 to 2001-01-01 so clearly there is no issue at 2000-01-02
1999 mid August to 2000 dataset has times again from 1999-01-01 to 2000-01-02??? I don't know how it even knows about the files earlier in 1999 than what I input.

import kerchunk.hdf
import fsspec
import pathlib
from kerchunk.combine import MultiZarrToZarr
import ujson

fs2 = fsspec.filesystem('')  #local file system to save final jsons to

so = dict(default_fill_cache=False, default_cache_type='first')

for u in urls:
    outf = out_base / pathlib.Path(u.stem.split(".")[-1]).with_suffix(".json")
    if not outf.exists():
        with fsspec.open(u, **so) as inf:
            h5chunks = kerchunk.hdf.SingleHdf5ToZarr(inf, str(u), inline_threshold=100)
            with fs2.open(outf, 'wb') as f:
                f.write(ujson.dumps(h5chunks.translate()).encode());

json_list = fs2.glob("ciofs_kerchunk2/1999*.json") # combine single json files
json_list += fs2.glob("ciofs_kerchunk2/2000*.json") # combine single json files

mzz = MultiZarrToZarr(
    json_list,
    concat_dims=["ocean_time"],
    identical_dims= ['lat_rho', 'lon_rho', "lon_psi", "lat_psi",
                    "lat_u", "lon_u", "lat_v", "lon_v",
                    "Akk_bak","Akp_bak","Akt_bak","Akv_bak","Cs_r","Cs_w",
                    "FSobc_in","FSobc_out","Falpha","Fbeta","Fgamma","Lm2CLM",
                    "Lm3CLM", "LnudgeM2CLM", "LnudgeM3CLM", "LnudgeTCLM",
                    "LsshCLM", "LtracerCLM", "LtracerSrc", "LuvSrc",
                    "LwSrc", "M2nudg", "M2obc_in", "M2obc_out", "M3nudg",
                    "M3obc_in", "M3obc_out", "Tcline", "Tnudg","Tobc_in", "Tobc_out",
                    "Vstretching", "Vtransform", "Znudg", "Zob", "Zos", "angle",
                    "dstart", "dt", "dtfast", "el", "f", "gamma2", "grid", "h",
                    "hc", "mask_psi", "mask_rho", "mask_u", "mask_v", "nHIS", "nRST",
                    "nSTA", "ndefHIS",  "ndtfast", "ntimes", "pm", "pn", "rdrg",
                    "rdrg2", "rho0", "spherical", "theta_b", "theta_s", "xl",
                    ],
)

# to keep in memory
out = mzz.translate()

import xarray as xr
ds = xr.open_dataset(
    "reference://", engine="zarr",
    backend_kwargs={
        "storage_options": {
            "fo": out,
        },
        "consolidated": False
    }
)

I recommend you run mzz.first_pass and have a look at the coords attribute to see what it populated there. If you have suggestions on how we can do more logging feedback, would happily oblige.

Ok thanks I'll take a detailed look there.

I understand the problem now.
ncdump -v ocean_time /mnt/vault/ciofs/HINDCAST/1999/axiom.ciofs.fields.hindcast.1999_0001.nc
gives
ocean_time = 3600, 7200, 10800, 14400, 18000, 21600, 25200, 28800, 32400,      36000, 39600, 43200, 46800, 50400, 54000, 57600, 61200, 64800, 68400,      72000, 75600, 79200, 82800, 86400 ;
and ocean_time has units ocean_time:units = "seconds since 1999-01-01 00:00:00" ;
and
ncdump -v ocean_time /mnt/vault/ciofs/HINDCAST/2000/axiom.ciofs.fields.hindcast.2000_0001.nc
gives
ocean_time = 3600, 7200, 10800, 14400, 18000, 21600, 25200, 28800, 32400,      36000, 39600, 43200, 46800, 50400, 54000, 57600, 61200, 64800, 68400,      72000, 75600, 79200, 82800, 86400 ;
and ocean_time has units ocean_time:units = "seconds since 2000-01-01 00:00:00" ;
Is there a way to account for this in the kerchunk set up?

I'm planning to fix this problem using nco if I can't get around it with kerchunk directly. If not, a warning that the concatenation variable units have to stay the same would be great.
As an example, the following modifies the units. I also need to go change the values:
ncap2 -O -s 'ocean_time@units="seconds since 1999-01-01 00:00:00"' axiom.ciofs.fields.hindcast.2000_0001.nc out.nc

Ah, good old C-F. kerchunk takes the actual values of the coordinates at face value unless you specify to try to use C-F decoding. In MultiZarrToZarr's doc on selectors:

        - "cf:{var}", interpret the value of var using cftime, returning a datetime.

          These will be automatically re-encoded with cftime, *unless* you specify an
          "M8[*]" dtype for the coordinate, in which case a conversion will be attempted.

@martindurant Awesome, this worked. For reference the MultiZarr command now looks like this and accurately maps time according to both their value and their unit:
json_list = fs2.glob("ciofs_kerchunk2/1999_0[2,3]??.json") # combine single json files
json_list += fs2.glob("ciofs_kerchunk2/2000*.json") # combine single json files
json_list = sorted(json_list)
mzz = MultiZarrToZarr(
    json_list,
    concat_dims=["ocean_time"],
    identical_dims= ['lat_rho', 'lon_rho', "lon_psi", "lat_psi",
                    "lat_u", "lon_u", "lat_v", "lon_v",
                    "Akk_bak","Akp_bak","Akt_bak","Akv_bak","Cs_r","Cs_w",
                    "FSobc_in","FSobc_out","Falpha","Fbeta","Fgamma","Lm2CLM",
                    "Lm3CLM", "LnudgeM2CLM", "LnudgeM3CLM", "LnudgeTCLM",
                    "LsshCLM", "LtracerCLM", "LtracerSrc", "LuvSrc",
                    "LwSrc", "M2nudg", "M2obc_in", "M2obc_out", "M3nudg",
                    "M3obc_in", "M3obc_out", "Tcline", "Tnudg","Tobc_in", "Tobc_out",
                    "Vstretching", "Vtransform", "Znudg", "Zob", "Zos", "angle",
                    "dstart", "dt", "dtfast", "el", "f", "gamma2", "grid", "h",
                    "hc", "mask_psi", "mask_rho", "mask_u", "mask_v", "nHIS", "nRST",
                    "nSTA", "ndefHIS",  "ndtfast", "ntimes", "pm", "pn", "rdrg",
                    "rdrg2", "rho0", "spherical", "theta_b", "theta_s", "xl",
                    ],
    coo_map = {"ocean_time": "cf:ocean_time"},
)

Thank you!!

Thanks @martindurant and @kthyng for this thread, has been very helpful as I've tried to work through a similar issue! I'm encountering a slightly different error when I try and use a similar argument for coo_map=. Running the mzz.first_pass() function gives me the error message below.
mzz.first_pass()

---------------------------------------------------------------------------
OverflowError                             Traceback (most recent call last)
Cell In[255], line 1
----> 1 mzz.first_pass()

File ~/micromamba/lib/python3.11/site-packages/kerchunk/combine.py:258, in MultiZarrToZarr.first_pass(self)
    256 z = zarr.open_group(fs.get_mapper(""))
    257 for var in self.concat_dims:
--> 258     value = self._get_value(i, z, var, fn=self._paths[i])
    259     if isinstance(value, np.ndarray):
    260         value = value.ravel()

File ~/micromamba/lib/python3.11/site-packages/kerchunk/combine.py:234, in MultiZarrToZarr._get_value(self, index, z, var, fn)
    232 units = datavar.attrs.get("units")
    233 calendar = datavar.attrs.get("calendar", "standard")
--> 234 o = cftime.num2date(o, units=units, calendar=calendar)
    235 if self.cf_units is None:
    236     self.cf_units = {}

File src/cftime/_cftime.pyx:617, in cftime._cftime.num2date()

File src/cftime/_cftime.pyx:414, in cftime._cftime.cast_to_int()

OverflowError: time values outside range of 64 bit signed integers

The units for the ocean_time variables are 'seconds since 1975-01-01 00:00:00', so there shouldn't be overlapping values for this dimension among different files. If I don't specify arguments to coo_map=, then I can only extract a dataset with 54 time-steps (should be ~30,000 from the nearly 10,000 JSON files) that all have bizarre values:
ds.ocean_time
array([-2.753701e+091, -2.897806e+079, -7.635061e-147,            nan,
        4.940656e-324,  7.905050e-323,  1.009376e-320,  5.215020e-310,
        5.215020e-310,  5.215020e-310,  5.215020e-310,  5.215020e-310,
        5.215020e-310,  6.951729e-310,  6.951729e-310,  6.951729e-310,
        6.951729e-310,  6.951729e-310,  6.951730e-310,  6.951730e-310,
        6.951730e-310,  6.951730e-310,  1.433150e-287,  1.458349e-287,
        1.483547e-287,  1.508745e-287,  1.533943e-287,  1.559141e-287,
        1.584339e-287,  1.609537e-287,  1.634735e-287,  1.659934e-287,
        1.685132e-287,  1.710330e-287,  1.735528e-287,  1.760726e-287,
        1.785924e-287,  1.811122e-287,  1.836320e-287,  1.861518e-287,
        1.886717e-287,  1.911915e-287,  1.937113e-287,  1.962311e-287,
        1.987509e-287,  2.012707e-287,  2.037905e-287,  2.063103e-287,
        2.088302e-287,  2.113500e-287,  2.138698e-287,  2.163896e-287,
        3.587487e+180,  5.812385e+180])

Would appreciate any suggestions about how to work through either of these issues, thanks so much!
For reference, here is my full code that I applied to create a set of JSON files and link them together after pulling NetCDFs from a remote repository.
from tempfile import TemporaryDirectory

import fsspec
import ujson
import xarray as xr
from kerchunk.combine import MultiZarrToZarr
from kerchunk.hdf import SingleHdf5ToZarr
from tqdm import tqdm

base_root = "s3://swem-scholarly-storage/hinson_data_archive_2023/Continuous/ChesROMS-ECB_Outputs/LongRun_"

# All directories from data archive
year_choices = ["1980_1984", "1985_1989", "1990_1994",
                "1995_1999", "2000_2004", "2005_2009",
                "2010_2014", "2015_2019", "2020_2024",
                "2025_2029", "2030_2034", "2035_2039",
                "2040_2044", "2045_2049", "2050_2054",
                "2055_2059", "2060_2064"]

# Grab all files from each 5-year directory
frange = range(0,61)
rfiles = []
for i in frange:
    iday = i + 1
    rfiles.append("roms_avg_" + f"{iday:04}" + ".nc")

base_urls = []
for ygrab in year_choices:
    for j in rfiles:
        base_urls.append(base_root + ygrab +"/" + j)

so = dict(mode="rb", anon=True, default_fill_cache=False, default_cache_type="first")
output_dir = "./"

# We are creating a temporary directory to store the .json reference
# files. Alternately, you could write these to cloud storage.
td = TemporaryDirectory()
temp_dir = td.name

# Use Kerchunk's `SingleHdf5ToZarr` method to create a `Kerchunk` index
# from a NetCDF file.

def generate_json_reference(u, temp_dir: str):
    with fs_read.open(u, **so) as infile:
        h5chunks = SingleHdf5ToZarr(infile, u, inline_threshold=300)
        fname = u.split("/")[-2] + "_" + u.split("/")[-1].strip(".nc")
        outf = f"{fname}.json"
        with open(outf, "wb") as f:
            f.write(ujson.dumps(h5chunks.translate()).encode())
        return outf

# Iterate through filelist to generate Kerchunked files
output_files = []
for fil in tqdm(base_urls):
    outf = generate_json_reference(fil, temp_dir)
    output_files.append(outf)

mzz = MultiZarrToZarr(
    output_files,
    concat_dims=["ocean_time"],
    identical_dims=["tracer", "boundary", "s_rho", "s_w", "eta_rho", "xi_rho",
                   "eta_u", "xi_u", "eta_v", "xi_v", "eta_psi", "xi_psi"],
    coo_map = {"ocean_time": "cf:ocean_time"},
)

The units for the ocean_time variables are 'seconds since 1975-01-01 00:00:00'

Can you verify that this is indeed the case throughout the dataset? My suspicion is that the epoch changes time to time, and you are seeing integer number of seconds somehow interpreted as floats. You should check that the times of the individual (pre-combine) datasets are interpreted correctly throughout the range before trying to convert them.

Thanks @martindurant for your response! I'm not sure that I can find discrepancies in the time units when they are read in versus those in the original file.
The NetCDF file shows this information in ncdump -h roms_avg_0001.nc
        double ocean_time(ocean_time) ;
                ocean_time:long_name = "averaged time since initialization" ;
                ocean_time:units = "seconds since 1975-01-01 00:00:00" ;
                ocean_time:calendar = "proleptic_gregorian" ;
                ocean_time:field = "time, scalar, series" ;

Below is the output from ncdump -v ocean_time roms_avg_0001.nc
 ocean_time = 157809600, 157896000, 157982400, 158068800, 158155200,
    158241600, 158328000, 158414400, 158500800, 158587200, 158673600,
    158760000, 158846400, 158932800, 159019200, 159105600, 159192000,
    159278400, 159364800, 159451200, 159537600, 159624000, 159710400,
    159796800, 159883200, 159969600, 160056000, 160142400, 160228800,
    160315200 ;

These values look fine for multiple files, all with the same attributes. Using python to read in a single file using the commands below, it looks to me like the additional attributes for ocean_time (e.g. units and calendar) are already accounted for, but aren't listed explicitly?
fs = s3fs.S3FileSystem(anon=True)

fileObj = fs.open(base_urls[0], mode="rb")
ds = xr.open_dataset(fileObj, engine='h5netcdf', chunks={'ocean_time': 1})

array(['1980-01-01T12:00:00.000000000', '1980-01-02T12:00:00.000000000',
       '1980-01-03T12:00:00.000000000', '1980-01-04T12:00:00.000000000',
       '1980-01-05T12:00:00.000000000', '1980-01-06T12:00:00.000000000',
       '1980-01-07T12:00:00.000000000', '1980-01-08T12:00:00.000000000',
       '1980-01-09T12:00:00.000000000', '1980-01-10T12:00:00.000000000',
       '1980-01-11T12:00:00.000000000', '1980-01-12T12:00:00.000000000',
       '1980-01-13T12:00:00.000000000', '1980-01-14T12:00:00.000000000',
       '1980-01-15T12:00:00.000000000', '1980-01-16T12:00:00.000000000',
       '1980-01-17T12:00:00.000000000', '1980-01-18T12:00:00.000000000',
       '1980-01-19T12:00:00.000000000', '1980-01-20T12:00:00.000000000',
       '1980-01-21T12:00:00.000000000', '1980-01-22T12:00:00.000000000',
       '1980-01-23T12:00:00.000000000', '1980-01-24T12:00:00.000000000',
       '1980-01-25T12:00:00.000000000', '1980-01-26T12:00:00.000000000',
       '1980-01-27T12:00:00.000000000', '1980-01-28T12:00:00.000000000',
       '1980-01-29T12:00:00.000000000', '1980-01-30T12:00:00.000000000'],
      dtype='datetime64[ns]')

Attributes:
long_name : averaged time since initialization
field : time, scalar, series

The ocean_time values that I find for any number of individual files look fine to me as well. Previously, I was using the code below to open multiple files and the ocean_time field worked without issue, it just takes a very long time and seems to fail with this many files.
fs = s3fs.S3FileSystem(anon=True)

fObjects = []

for kurl in base_urls:
    fileObj = fs.open(kurl, mode="rb")
    fObjects.append(fileObj)

ds = xr.open_mfdataset(fObjects, engine='h5netcdf', chunks={'ocean_time': 1})

Are these files public?

Yes, they're all publicly available here under Continuous/ChesROMS-ECB_Outputs/

Trying it with just the first two files produced no problem
out = kerchunk.hdf.SingleHdf5ToZarr("roms_avg_0001.nc").translate()
out2 = kerchunk.hdf.SingleHdf5ToZarr("roms_avg_0002.nc").translate()
mzz = MultiZarrToZarr(
    [out, out2],
    concat_dims=["ocean_time"],
    identical_dims=["tracer", "boundary", "s_rho", "s_w", "eta_rho", "xi_rho",
                   "eta_u", "xi_u", "eta_v", "xi_v", "eta_psi", "xi_psi"],
    coo_map = {"ocean_time": "cf:ocean_time"},
)
final = mzz.translate()
ds = xr.open_dataset(final, engine="kerchunk")

producing ocean_time
<xarray.DataArray 'ocean_time' (ocean_time: 60)>
array(['1980-01-01T12:00:00.000000000', '1980-01-02T12:00:00.000000000',
       '1980-01-03T12:00:00.000000000', '1980-01-04T12:00:00.000000000',
       '1980-01-05T12:00:00.000000000', '1980-01-06T12:00:00.000000000',
...

Adding in the first file from the second tranche (1985) still works ok, despite the gap in the timeline.

Thanks Martin for testing this! Your code also works fine for me if I download the files to my machine first. Is SingleHdf5ToZarr still intended to work with the JSON files that I've created? Below is some code that I've modified based on @kthyng's example. This still throws me the error OverflowError: time values outside range of 64 bit signed integers.
fs2 = fsspec.filesystem('')  #local file system to save final jsons to

so = dict(default_fill_cache=False, default_cache_type='first')

for u in base_urls:
    outf = pathlib.Path(u.split("/")[-2] + "_" + u.split("/")[-1].strip(".nc")).with_suffix(".json")
    if not outf.exists():
        with fsspec.open(u, **so) as inf:
            h5chunks = kerchunk.hdf.SingleHdf5ToZarr(inf, str(u), inline_threshold=100)
            with fs2.open(outf, 'wb') as f:
                f.write(ujson.dumps(h5chunks.translate()).encode());

json_list = fs2.glob("LongRun*.json") # combine single json files

# Concatenate JSON files
mzz = MultiZarrToZarr(
    json_list,
    concat_dims=["ocean_time"],
    identical_dims=["tracer", "boundary", "s_rho", "s_w", "eta_rho", "xi_rho",
                   "eta_u", "xi_u", "eta_v", "xi_v", "eta_psi", "xi_psi"],
    coo_map = {"ocean_time": "cf:ocean_time"},
)
final = mzz.translate()
ds = xr.open_dataset(final, engine="kerchunk")

Is there something in my code that could be altering ocean_time when I try and create the JSON files? Or is this the wrong way to go about creating the JSON files that I then feed into the MultiZarrToZarr function?

examples.zip
Not sure how helpful this will be, but here are two example scripts I have for creating kerchunk files for two separate ROMS ocean models.

Is SingleHdf5ToZarr still intended to work with the JSON files that I've created?

Yes, it should work whether the source file or reference file is local or remote.

Is there something in my code that could be altering ocean_time

I don't immediately see anything. You should look at the references "ocean_time/0", "ocean_time/.zattrs", "ocean_time/.zarray" for some of the reference sets to see if there's a discrepancy.
I notice you have inline_threshold set, so the data reference ("/0") should be some bytes that evaluates to ~ints like 157809600~ ; they are floats "<f8", not ints. I wonder if the "cf:" mapper assumes they are int.
Also, the chunk size is (512, ), but only 30 values are valid values, shape=(30,), and the rest of the data is filled with 9.96920997e+36. I'm not sure if this is a problem - it still worked for me - but it's certainly it's odd.

Thanks @kthyng and @martindurant for your helpful responses! I tried implementing some of the example code from @kthyng but ran into the error NoCredentialsError: Unable to locate credentials when trying to implement the code below, specifically for the line  with fsspec.open(u, **so) as inf:
for u in base_urls:
    outf = pathlib.Path(u.split("/")[-2] + "_" + u.split("/")[-1].strip(".nc")).with_suffix(".json")
    if not outf.exists():
        with fsspec.open(u, **so) as inf:
            h5chunks = kerchunk.hdf.SingleHdf5ToZarr(inf, str(u), inline_threshold=200)
            with fs2.open(outf, 'wb') as f:
                f.write(ujson.dumps(h5chunks.translate()).encode());

However, I was able to make more headway by following some of the steps from the kerchunk medium tutorial. The code below was able to rapidly (within ~5 minutes) open the list of JSONs and then read them in as a combined dataset without any problems in concatenating along the time dimension.
fname_overall_kerchunk = "ciofs_kerchunk.json"
json_list = fs2.glob("LongRun*.json") # combine single json files

m_list = []
for j in tqdm(json_list):
    with open(j) as f:
        m_list.append(fsspec.get_mapper("reference://",
                        fo=ujson.load(f),
                        remote_protocol='s3',
                        remote_options={'anon':True}))

ds = xr.open_mfdataset(m_list, engine='zarr', combine='nested', concat_dim='ocean_time',
                        coords='minimal', data_vars='minimal', compat='override',
                        parallel=True)

I appreciate all your suggestions in helping me figure out this error, this solution is now working well for me!

You may want to talk with @TomNicholas about why xarray can concat your data but MZZ apparently can't. Since I never got as far as your problem, I can't say more, but if having xarray o the concat for you on every load is good enough for you, that's fine.

@kyle-hinson @kthyng you should now be able to do this with VirtualiZarr, since https://github.com/zarr-developers/VirtualiZarr/pull/122 was merged. If you do
import glob
import xarray as xr
from virtualizarr import open_virtual_dataset

vds_list = [
    open_virtual_dataset(file, loadable_variables=['time'], cftime_variables=['ocean_time'], indexes={})
    for file in glob.glob("LongRun*.json")
]

vds = xr.combine_nested(
    vds_list,
    concat_dim='ocean_time',
    coords='minimal',
    data_vars='minimal',
    compat='override',
)

vds.virtualize.to_kerchunk(...)

then you will literally be using xarray to handle the concatenation.


[Bug]:  Bug when trying to use FastEmbedEmbeddings()

What happened?
I''ve been using FastEmbedEmbeddings for the past few weeks and yesterday it stopped working and I don't know why. Below are pictures of the error messages and my code.

What Python version are you on? e.g. python --version
Python 3.10
Version
0.2.7 (Latest)
What os are you seeing the problem on?
Windows
Relevant stack traces and/or logs
No response

Same error with python 3.9 and latest versions of both langchain-community and fastembed. Any update on this?

Hey guys. The issue is found. There's a PR to fix it and also update the integration.

Can you try updating Langchain Community and FastEmbed to latest?
I tried that, but it didn't work. I figured out the solution: I needed to install my pip dependencies in a specific order. Thank you for your help!


aws_cloudwatch_scaler - http invalid header field value

Hello, after deploying the autoscaler based on aws-cloudwatch sqs number of messages, the TARGET column on kubectl get hpa shows <unknown>/3 and no deployment scales.
On the keda-operator log this error is shown so I guess it's something related to that:
{"level":"error","ts":1598467409.1905313,"logger":"aws_cloudwatch_scaler","msg":"Failed to get output","error":"RequestError: send request failed\ncaused by: Post \"https://monitoring.us-east-1.amazonaws.com/\": net/http: invalid header field value \"AWS4-HMAC-SHA256 Credential=XXXXXXXXXXXXXXXXXXXXX/20200826/us-east-1/monitoring/aws4_request, SignedHeaders=content-length;content-type;host;x-amz-date, Signature=XXXXXXXXXXXXXXXXXXXXX\" for key Authorization","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\t/Users/zroubali/go/pkg/mod/github.com/go-logr/zapr@v0.1.1/zapr.go:128\ngithub.com/kedacore/keda/pkg/scalers.(*awsCloudwatchScaler).GetCloudwatchMetrics\n\tkeda/pkg/scalers/aws_cloudwatch_scaler.go:249\ngithub.com/kedacore/keda/pkg/scalers.(*awsCloudwatchScaler).IsActive\n\tkeda/pkg/scalers/aws_cloudwatch_scaler.go:185\ngithub.com/kedacore/keda/pkg/handler.(*ScaleHandler).handleScaleDeployment\n\tkeda/pkg/handler/scale_loop.go:119\ngithub.com/kedacore/keda/pkg/handler.(*ScaleHandler).handleScale\n\tkeda/pkg/handler/scale_loop.go:45\ngithub.com/kedacore/keda/pkg/handler.(*ScaleHandler).HandleScaleLoop\n\tkeda/pkg/handler/scale_loop.go:28"}
Expected Behavior
Deployment scales and kubectl get hpa shows correct info.
Actual Behavior
kubectl get hpa shows <unknown>/3 and no deployment scales.
Steps to Reproduce the Problem
Here is my configuration file:
apiVersion: v1
kind: Secret
metadata:
  name: aws-secrets
data:
  AWS_ACCESS_KEY_ID: XXXXXXXXXXXXXX
  AWS_SECRET_ACCESS_KEY: XXXXXXXXXXXXXXXXXXX
---
apiVersion: keda.k8s.io/v1alpha1
kind: TriggerAuthentication
metadata:
  name: keda-trigger-auth-aws-credentials
spec:
  secretTargetRef:
  - parameter: awsAccessKeyID     # Required.
    name: aws-secrets        # Required.
    key: AWS_ACCESS_KEY_ID        # Required.
  - parameter: awsSecretAccessKey # Required.
    name: aws-secrets        # Required.
    key: AWS_SECRET_ACCESS_KEY    # Required.
---
apiVersion: keda.k8s.io/v1alpha1
kind: ScaledObject
metadata:
  name: worker-sqs-queue-scaledobject
  labels:
    tier: api
spec:
  scaleTargetRef:
    deploymentName: worker
  triggers:
  - type: aws-cloudwatch
    metadata:
      namespace: AWS/SQS
      dimensionName: QueueName
      dimensionValue: staging-queue
      metricName: ApproximateNumberOfMessagesVisible
      targetMetricValue: "3"
      minMetricValue: "0"
      awsRegion: "us-east-1"
    authenticationRef:
      name: keda-trigger-auth-aws-credentials

Specifications

KEDA Version:

Client Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.6", GitCommit:"dff82dc0de47299ab66c83c626e08b245ab19037", GitTreeState:"clean", BuildDate:"2020-07-15T16:58:53Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"16", GitVersion:"v1.16.13", GitCommit:"1da71a35d52fa82847fd61c3db20c4f95d283977", GitTreeState:"clean", BuildDate:"2020-07-15T21:59:26Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"}

Platform & Version:
Azure Kubernetes Services
Kubernetes Version:
1.16.13
Scaler(s):
AWS CloudWatch

Same issue here.
Installed everything yesterday with helm.
{"level":"error","ts":1599157802.9904482,"logger":"aws_cloudwatch_scaler","msg":"Failed to get output","error":"RequestError: send request failed\ncaused by: Post \"https://monitoring.us-east-1.amazonaws.com/\": net/http: invalid header field value \"AWS4-HMAC-SHA256 Credential=\\xxx\\xxx\\xxxx]\\xxx\\xxx\\xxxxx\\xxxxx/xxxxx/xxxxx/monitoring/aws4_request, SignedHeaders=content-length;content-type;host;x-amz-date, Signature=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx\" for key Authorization","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\t/Users/zroubali/go/pkg/mod/github.com/go-logr/zapr@v0.1.1/zapr.go:128\ngithub.com/kedacore/keda/pkg/scalers.(*awsCloudwatchScaler).GetCloudwatchMetrics\n\tkeda/pkg/scalers/aws_cloudwatch_scaler.go:249\ngithub.com/kedacore/keda/pkg/scalers.(*awsCloudwatchScaler).IsActive\n\tkeda/pkg/scalers/aws_cloudwatch_scaler.go:185\ngithub.com/kedacore/keda/pkg/handler.(*ScaleHandle

@engyishere I had to deploy ScaledObject with
  triggers:
  - type: aws-cloudwatch
    metadata:
      namespace: AWS/SQS
      dimensionName: QueueName
      dimensionValue: staging-queue
      metricName: ApproximateNumberOfMessagesVisible
      metricStat: Maximum
      metricStatPeriod: "30"
      targetMetricValue: "200"
      minMetricValue: "1"
      awsRegion: "us-east-1"
      awsAccessKeyID: AWS_ACCESS_KEY_ID
      awsSecretAccessKey: AWS_SECRET_ACCESS_KEY

These AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY are the env variable names on the scaleTargetRef.deploymentName deployment. For example:
apiVersion: apps/v1
kind: Deployment
metadata:
  name: php-deployment
...
...
        env:
        - name: AWS_ACCESS_KEY_ID
          value: XXXXXXXXXXXXXXXX
        - name: AWS_SECRET_ACCESS_KEY
          value: XXXXXXXXXXXXXX

Hope this helps

@cachila  Thanks for taking time to get back to me. Are you using v2  for deployment of keda or 1.5.0.
This is the changes i have made and i am currently using v2  to test the new release if that resolves the issue. if possible send the files please.
`apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
name: aws-cloudwatch-queue-scaledobject
namespace: default
labels:
app: engy-worker-edo
spec:
scaleTargetRef:
name: engy-worker-edo
deploymentName: engy-worker-edo
pollingInterval: 10
cooldownPeriod: 300
minReplicaCount: 3
maxReplicaCount: 8
triggers:

type: aws-cloudwatch
metadata:
namespace: AWS/SQS
dimensionName: QueueName
dimensionValue: keda
metricName: ApproximateNumberOfMessagesVisible
targetMetricValue: "20"
minMetricValue: "0"
awsRegion: "us-east-1"
awsAccessKeyID: AWS_ACCESS_KEY_ID
awsSecretAccessKey: AWS_SECRET_ACCESS_KEY
authenticationRef:
name: keda-trigger-auth-aws-credentials`

forgot to mention that AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY  are defined as env variables in deployment .

This is the ScaledObject file. Is this what you want me to share?
apiVersion: keda.k8s.io/v1alpha1
kind: ScaledObject
metadata:
  name: worker-scaled-object
  labels:
    deploymentName: php-deployment
spec:
  scaleTargetRef:
    deploymentName: worker # must be in the same namespace as the ScaledObject
  pollingInterval: 30
  cooldownPeriod: 90
  minReplicaCount: 1
  maxReplicaCount: 100
  triggers:
  - type: aws-cloudwatch
    metadata:
      namespace: AWS/SQS
      dimensionName: QueueName
      dimensionValue: staging-queue
      metricName: ApproximateNumberOfMessagesVisible
      metricStat: Maximum
      metricStatPeriod: "30"
      targetMetricValue: "200"
      minMetricValue: "1"
      awsRegion: "us-east-1"
      awsAccessKeyID: AWS_ACCESS_KEY_ID
      awsSecretAccessKey: AWS_SECRET_ACCESS_KEY

Not working for me. Here's what i am doing.

helm to install keda.
Created a secret to store AWS Creds.
Created a deployment with env

apiVersion: apps/v1
kind: Deployment
metadata:
  name: ehush-worker-edo
  labels:
    app: ehush
spec:
  selector:
    matchLabels:
      app: ehush
  template:
    metadata:
      labels:
        app: ehush
    spec:
      containers:
      - name: ehush
        image: nginx:latest
        resources:
          requests:
            memory: "128Mi"
            cpu: "200m"
        env:
        - name: AWS_ACCESS_KEY_ID
          valueFrom:
            secretKeyRef:
              name: keda-aws-secrets
              key: AWS_ACCESS_KEY_ID
        - name: AWS_SECRET_ACCESS_KEY
          valueFrom:
            secretKeyRef:
              name: keda-aws-secrets
              key: AWS_SECRET_ACCESS_KEY

Created ScaledObject , no trigger authentication. scaledobject file as you have provided.

keda-hpa-ehush-worker-edo   Deployment/ehush-worker-edo   <unknown>/20 (avg)   1         10        1          11m

HPA creates the minimum required pod but there is not scaling whatsoever. In keda operator logs, no mention of  polling Cloudwatch metrics.
Is there something i am missing? Does your setup work properly with cloudwatch metrics?

Update: Trigger for AWS SQS working just fine but for Cloudwatch still an issue.
If possible, please attach all files for me to compare. maybe i am still missing something.

I'm not using a Secret, just the file I posted previously and a Deployment pretty much like yours but env values directly there like I posted before.
If AWS SQS is working, I have no clue why Cloudwatch is not.

SQS is not working here neither for me :( same issue, deployed as shown and it posts  Post \"https://sqs.eu-west-1.amazonaws.com/\": net/http: invalid header field value \"AWS4-HMAC-SHA256 Credential

@XavierGeerinck is this something that was fixed by #1332 and we can close this one?

@zroubalik no this is unrelated sadly enough. SQS doesn't work at all for me, the bug fix I did was something I just found... when utilizing the SQS scaler the error as shown in my previous comment shows up.

@XavierGeerinck I've got the same errors. There was a mistake with encoding of secrets: the trailed '\n' has been included in the hashed value.
Everything has become well when I hashed values correctly:
echo -n 'AKIASOMEAWSACCESSKEID'  | base64

I'm unable to repro this using the following
apiVersion: v1
kind: Secret
metadata:
  name: test-secrets
  namespace: keda-test
stringData:
  AWS_ACCESS_KEY_ID: <value>
  AWS_SECRET_ACCESS_KEY: <value>
---
apiVersion: keda.sh/v1alpha1
kind: TriggerAuthentication
metadata:
  name: keda-trigger-auth-aws-credentials
  namespace: keda-test
spec:
  secretTargetRef:
  - parameter: awsAccessKeyID     # Required.
    name: test-secrets            # Required.
    key: AWS_ACCESS_KEY_ID        # Required.
  - parameter: awsSecretAccessKey # Required.
    name: test-secrets            # Required.
    key: AWS_SECRET_ACCESS_KEY    # Required.
---
apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: aws-sqs-queue-scaledobject
  namespace: keda-test
spec:
  scaleTargetRef:
    name: sqs-worker
  pollingInterval: 10
  cooldownPeriod: 20
  triggers:
  - type: aws-cloudwatch
    authenticationRef:
      name: keda-trigger-auth-aws-credentials
    metadata:
      namespace: AWS/SQS
      dimensionName: QueueName
      dimensionValue: ahmels-sqs-keda
      metricName: ApproximateNumberOfMessagesVisible
      metricStat: Maximum
      metricStatPeriod: "30"
      targetMetricValue: "200"
      minMetricValue: "1"
      awsRegion: "us-west-2"
      # remove authenticationRef if using this method.
      # awsAccessKeyIDFromEnv: AWS_ACCESS_KEY_ID
      # awsSecretAccessKeyFromEnv: AWS_SECRET_ACCESS_KEY
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: sqs-worker
  namespace: keda-test
  labels:
    app: sqs-worker
spec:
  selector:
    matchLabels:
      app: sqs-worker
  template:
    metadata:
      labels:
        app: sqs-worker
    spec:
      containers:
      - name: sqs-worker
        image: docker.io/ahmelsayed/sqs:1
        resources:
          requests:
            memory: "128Mi"
            cpu: "200m"
        env:
        - name: AWS_ACCESS_KEY_ID
          valueFrom:
            secretKeyRef:
              name: test-secrets
              key: AWS_ACCESS_KEY_ID
        - name: AWS_SECRET_ACCESS_KEY
          valueFrom:
            secretKeyRef:
              name: test-secrets
              key: AWS_SECRET_ACCESS_KEY

note that awsAccessKeyID and awsSecretAccessKey are now awsAccessKeyIDFromEnv and awsSecretAccessKeyFromEnv. See v2 breaking change here https://github.com/kedacore/keda/releases/tag/v2.0.0
I'll close this issue based on that.


Fixing typo on actual API to use

Signed-off-by: Matthias Wessendorf<EMAIL_ADDRESS>Proposed Changes

update a deprecated API typo, since the actual referenced API was renamed during the PR

Release Note

/assign @skonto
/assign @pierDipi
/assign @houshengbo

/retest


IOS - Can not send data/Image/text when press send button of share popup

Hi guys, i got problem with this framework, can you spend some time to help me?
Please see my video first.
https://youtu.be/7_9F0RTQX6M
I add react-native-share-menu and import all of things need to install framework. It 's ok with android but in ios, when i press post button, data seem not send to my app.
So everyone can help me, i need to send data to my app ios same as android version

@jdmunro  @edbice @caiosba

@dungdt-itealteam you can use the documentation Custom View (https://github.com/meedan/react-native-share-menu/blob/master/SHARE_EXTENSION_VIEW.md).
You can also follow the example app which is complete.

Thank for reply @ilenin. But after all step, when i success show custom share view and see the data display in custom share view of extension, when i press continue in app, i can not get any data. Can you tell me any important  note for this step. Thanks


training from scratch

Hi, is it possible to train the model? I could not find any code to train model.
I am wondering how to create the pre-trained model (UCSD Ped2, CUHK Avenue, ShanghaiTech Campus). Is this repository inference only?
Thank you.

Hi, is it possible to train the model? I could not find any code to train model.
I am wondering how to create the pre-trained model (UCSD Ped2, CUHK Avenue, ShanghaiTech Campus). Is this repository inference only?
Thank you.

I have the same question with you, have you already resovled it?

Hi, is it possible to train the model? I could not find any code to train model.
I am wondering how to create the pre-trained model (UCSD Ped2, CUHK Avenue, ShanghaiTech Campus). Is this repository inference only?
Thank you.

Hi, thank you for your interest!
I don't have permission to release the training code of this project. However, I can contribute to your repository.

Hi, thank you for your interest!
I don't have permission to release the training code. However, I can contribute to your repository if you need help.


feat (scipy.special): Add a xla version of scipy.special.gamma function

Add gamma fcn api in scipy.special
Add tests for this purpose
Add function to the docs

Currently, there is no implementation of the gamma function in jax but there is one in scipy.special. This breaks some higher level jit-compilation like in the blackjax backend for pymc. This commit adds the missing gamma function.
Resolves: #15409
See also the previously failed PR: #15652

Hi - sorry we haven't been able to get this merged. The additional test case is affecting the random seed for other tests, which leads to inputs for special.i0e that lead to test failures. The source of the issue in i0e is unclear, so I'm not sure how to proceed.

Ah, looks like the problem is in the check_grads test for i0e with entries near zero. Because the function has a discontinuity at zero, the numerical gradient is inaccurate and causes the test to fail. I'm trying out a fix...

Once #15673 is in we should be able to merge this.

Thank you very much @jakevdp :-)


GH 1766 Link to different channel from RHS run details

Summary

Added context menu item to link to different channel to RHS run details.
Also fixed bug related to moving the same run multiple times without reloading.
Ticket Link

Fixes #1766
Checklist

[ ] Telemetry updated
[ ] Gated by experimental feature flag
[x] Unit tests updated

/update-branch

This is a solid change, but likely not something that will fit into the upcoming priorities shift, so closing out.


add links, remove repo list

add link to repo overview of near github organization
remove repo list (in favor or link to overview)
add link to all "goog first issues"

thanks for this @lazaridiscom!  appreciate the call out to good first issues and the top level github repo
regarding

remove repo list

i think it's useful to have this list here, isn't it?  otherwise newcomers are looking at over 100 repos and scanning the top of the list for most recent activity to get a sense of the possible scope of work.

remove repo list (in favor or link to overview)

https://github.com/nearprotocol
Contains pinned repos and repo list => all a visitor needs
no need to duplicate info and increase tho doc size unnecessarily.

no need to duplicate info and increase tho doc size unnecessarily.

while i don't think size of the document is important in this case, duplicating the list of key repos from pinned repos does seem like maintenance overhead, true.
but GitHub only lets us pin 6 repos to the top of the org landing page and we have more than 6 repos that we want to introduce to potential contributors.
let's try merging your changes for now and collect feedback.
thanks for the edits!

while i don't think size of the document is important in this case,

Keep docs compact, especially if they target professional folks. No need to exhaust them with redundant (possibly patronizing) talk.
If you want to address more casual contributors, possibly ones who try to contribute for the first time, write an additional intro and link to it. "For beginner contributors". The risk here is that later on, the team has to deal with beginner-coders (e.g. a blockchain project is not the best code-base to start a contributor career).


Adding DCO check to repo

opensearch-security pull request intake form
Please provide as much details as possible to get feedback/acceptance on your PR quickly

Category: (Enhancement, New feature, Bug fix, Test fix, Refactoring, Maintenance, Documentation)

Enhancement

Github Issue # or road-map entry, if available:

Closes #1463

Description of changes:

Added DCO check to repository

Why these changes are required?

See https://github.com/opensearch-project/project-meta/issues/17

What is the old behavior before changes and new behavior after changes? (Please add any example/logs/screen-shot if available)

No DCO checks on PR

Testing done: (Please provide details of testing done: Unit testing, integration testing and manual testing)

N/A

TO-DOs, if any: (Please describe pending items and provide Github issues# for each of them)

N/A

Is it backport from main branch? (If yes, please add backport PR # and commits #)

No
By making a contribution to this project, I certify that:
(a) The contribution was created in whole or in part by me and I
have the right to submit it under the open source license
indicated in the file; or
(b) The contribution is based upon previous work that, to the best
of my knowledge, is covered under an appropriate open source
license and I have the right under that license to submit that
work with modifications, whether created in whole or in part
by me, under the same open source license (unless I am
permitted to submit under a different license), as indicated
in the file; or
(c) The contribution was provided directly to me by some other
person who certified (a), (b) or (c) and I have not modified
it.
(d) I understand and agree that this project and the contribution
are public and that a record of the contribution (including all
personal information I submit with it, including my sign-off) is
maintained indefinitely and may be redistributed consistent with
this project or the open source license(s) involved.
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Codecov Report

Merging #1468 (53f0444) into main (6436c15) will increase coverage by 0.01%.
The diff coverage is n/a.

:exclamation: Current head 53f0444 differs from pull request most recent head 6a18adc. Consider uploading reports for the commit 6a18adc to get more accurate results

@@             Coverage Diff              @@
##               main    #1468      +/-   ##
============================================
+ Coverage     64.88%   64.89%   +0.01%
- Complexity     3228     3229       +1
============================================
  Files           247      247
  Lines         17335    17335
  Branches       3072     3072
============================================
+ Hits          11247    11249       +2
+ Misses         4532     4531       -1
+ Partials       1556     1555       -1

Impacted Files
Coverage Δ

.../dlic/auth/ldap2/LDAPConnectionFactoryFactory.java
58.01% <0.00%> (+1.52%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6436c15...6a18adc. Read the comment docs.


Call Depth Overflow During Publish

Describe the bug
In v0.96.0, when publishing an ADF via Publish-AdfV2FromJson, the deleting of duplicate LinkedService causes an overflow crash.
To Reproduce

Create component in an existing ADF which has a duplicate id / name using the Azure ADF UI.
Export the definition of the entire ADF via Git.
Confirm the duplicate component throws a warning via Test-AdfCode.
Try to publish to the same ADF via Publish-AdfV2FromJson, while removing both instances of the duplicate component using DeleteNotInSource:

$Option = New-AdfPublishOption;
$Option.DeleteNotInSource = $true;
$Option.DoNotDeleteExcludedObjects = $false;

Publish-AdfV2FromJson `
    -RootFolder $RootFolder `
    -ResourceGroupName $ResourceGroupName `
    -DataFactoryName $DataFactoryName `
    -Stage $DeployEnv `
    -Location $Location `
    -Method $Method `
    -Option $Option;

Observe that the unused resources in the ADF will be deleted successfully, except once the duplicate component is hit.  The duplicate component is the one that will cause the overflow error.

Removing object: [LinkedService].[bdas_dw]
Removing object: [LinkedService].[bdas_dw]
Removing object: [LinkedService].[bdas_dw]
Removing object: [LinkedService].[bdas_dw]
Removing object: [LinkedService].[bdas_dw]
Removing object: [LinkedService].[bdas_dw]
Removing object: [LinkedService].[bdas_dw]
Removing object: [LinkedService].[bdas_dw]
Removing object: [LinkedService].[bdas_dw]
Removing object: [LinkedService].[bdas_dw]
Removing object: [LinkedService].[bdas_dw]
Removing object: [LinkedService].[bdas_dw]
Removing object: [LinkedService].[bdas_dw]
Removing object: [LinkedService].[bdas_dw]
Removing object: [LinkedService].[bdas_dw]
Removing object: [LinkedService].[bdas_dw]
Removing object: [LinkedService].[bdas_dw]
Remove-AdfObject: /usr/local/share/powershell/Modules/azure.datafactory.tools/0.96.0/private/Remove-AdfObject.ps1:113
Line |
 113 |              Remove-AdfObject -adfSource $adfSource -obj $_ -adfInstan …
     |              ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     | The script failed due to call depth overflow.

##[error]PowerShell exited with code '1'.

I inherited this existing ADF on a project.  When I exported it to Git and ran the Test-AdfCode I noticed the Warning.  Lo and behold, the overflow is on the same component that threw the warning.

I'm seeing the same behavior.

Check whether you have another object in ADF called bdas_dw?

Ah - on my end - I have two objects in ADF with the same name - and I'm trying to delete both of them. Would that explain it ?

Yes, there is a duplicate in the ADF, which is allowed by Azure, but this tool cannot delete it.  It is a defect in this tool IMO.  It should handle the remote deleting of duplicates gracefully, and not crash.

I don't know why Azure allows duplicates, but it does.

Check whether you have another object in ADF called bdas_dw?

Ah - on my end - I have two objects in ADF with the same name - and I'm trying to delete both of them. Would that explain it ?

yes, this tool can't handle that scenario apparently


Add EqP and OrdP

What you mean by I still think this is bad due to a lack of cannonicity. Can you give an example?

Actually as just adding this super class is a breaking change, it would be nice to cut a release before it. The breakage will be annoying for us, and the only real beneficiary is StableName which is quite obscure. (Const and IORef I think are better handled with just GCoercible.)

Actually as just adding this super class is a breaking change, it would be nice to cut a release before it.

I'm not making release anytime soon. Probably with QuantifiedConstraints.
HEq is enough to write HEq f => Eq (Some f) instance. It must be good. The current GEq with type-equality is too strong of requirement. GCoerce as well.

only real beneficiary is StableName which is quite obscure.

It's not the only one.
E.g. data Fin n where FZ :: Fin (S n); FS :: Fin n -> Fin (S n) can have EqP instance. Also data Expr (ctx :: Ctx) where ... can be structurally compared even if contexts don't match, and Name (ctx :: Ctx) as in https://arxiv.org/abs/2210.04729
But for now this change doesn't add superclass relations, it only breaks the Eq and Ord instances for Some.
(And the motivation is to be able to compare Some Fin or Some Name).
The migration preparation for addition of super-class EqP f => GEq f is mentioned in the changelog.

-- with GHC-9.4
{-# LANGUAGE GADTs, QuantifiedConstraints #-}

import Data.Kind
import Data.Type.Equality

data Foo :: Type -> Type where
  MkFoo :: Int -> Foo Int

instance Eq a => Eq (Foo a) where
  MkFoo x == MkFoo y = x == y

class (forall a. Eq a => Eq (f a)) => Eq1 f where
  liftEq :: (a -> b -> Bool) -> f a -> f b -> Bool

instance Eq1 Foo where
  liftEq eq (MkFoo x) (MkFoo y) = eq x y

-- so far so good.

-- **BUT**, if GEq gets quantified superclass:

class (forall a. Eq (f a)) => GEq f where
  geq :: f a -> f b -> Maybe (a :~: b)

{-
instance GEq Foo where

will error with

    • No instance for (Eq a)
        arising from the superclasses of an instance declaration

Therefore Foo as above is not counter-example for not having (moral)
Eq1 as a moral superclass for EqP and GEq.

-}

-- If instead we had

data Bar :: Type -> Type where
  MkBar :: Int -> Bar Int

instance Eq (Bar a) where
  MkBar x == MkBar y = x == y

instance Eq1 Bar where
  liftEq _ (MkBar x) (MkBar y) = x == y

-- then we can have
instance GEq Bar where
  geq (MkBar x) (MkBar y) = if x == y then Just Refl else Nothing

-- and
class (forall a. Eq (f a)) => EqP f where
  eqp :: f a -> f b -> Bool

instance EqP Bar where
  eqp (MkBar x) (MkBar y) = x == y

instance EqP Proxy where
    eqp _ _ = True

Seems really suspicious to me. When would I want proxies at different types to compare equal?
Do we really need these classes? What's wrong with just using liftEq (\_ _ -> True) or liftCompare (\_ _ -> EQ) for your use-case? I'm already kind of suspicious about Eq1's right to exist apart from being convenience-machinery for defining Eq instances, and this seems to split hairs further in a way that I don't quite grasp.

@cgibbard you cannot write Eq1 instance for SNat. @Ericson2314  also gives an example where EqP and Eq1 can be different (and I fail to find a sound reason to rule out that example).
EqP is term comparison. The use case is to define Eq (Some f) instance. Some Proxy ~~ (), or do you disagree?

Eq1's right to exist apart

The heterogenity of liftEq
liftEq :: (a -> b -> Bool) -> f a -> f b -> Bool

is useful. That's not expressible with just forall a. Eq a => Eq (f a).
Also Eq1 f => Ord1 f is superclass constraint is strong requirement, as forall a. Ord a => Ord (f a) does not imply forall a. Eq a => Eq (f a) (variance in play!).

I would usually consider Some Proxy to have a value for each type, but as we've explicitly hidden the type from view, given two terms of type Some Proxy, it's no longer possible to decide if they're equal.
For an analogy which makes this somewhat clear in Coq:
Inductive Proxy (a : Type) : Type :=
  | MkProxy : Proxy a.

Inductive Some (f : Type -> Type) : Type :=
  | MkSome : forall a, f a -> Some f.

Proposition SomeProxyIsProp : forall (x y : Some Proxy), x = y.
  intros x y.
  destruct x.
  destruct y.
  <unfinished proof>

At this point in trying to show that values of type Some Proxy are unique, we have this context:
1 subgoal
a : Type
p : Proxy a
a0 : Type
p0 : Proxy a0
______________________________________(1/1)
MkSome Proxy a p = MkSome Proxy a0 p0

We're stuck here because we have no evidence that a = a0, so p = p0 won't even typecheck.

Note that we do have the following, that for each type a, values of type Proxy a are unique:
Proposition ProxyIsProp : forall a, forall (x y : Proxy a), x = y.
  intros a x y.
  destruct x.
  destruct y.
  reflexivity.
Qed.

@cgibbard I see your point, but I disagree. I don't see much value in restricting Eq Some to that kind of (almost) structural equality. Having EqP f => Eq (Some f) would allow to compare Some StableName too.

I reverted the change of Some instances. There might be code which has GEq f => ... constraints and uses Eq (Some f). Breaking that implication may introduce unfortunate middle state which is not trivial to workaround.
So the EqP f => Eq (Some f) instance change and EqP f => GEq f change will come at the some time, some time in the future.

I also added a compatibility law for EqP, which is similar to one [Eq](https://hackage.haskell.org/package/base-<IP_ADDRESS>/docs/Data-Eq.html#t:Eq] has
eqp x y ≡ True ⇒ f x == f y ≡ True -- for polymorphic f :: forall x. f x -> a and Eq a

This makes Eq (Some f) instance fully lawful, as Eq compatibility is implied by EqP compatibility. This also addresses @cgibbard concern, as there isn't f :: forall x. Proxy x -> a which isn't a constant functions (parametricity), therefore EqP Proxy instance is not problematic: Coq variant of Some @Proxy @x Proxy and Some @Proxy @y Proxy aren't equal, but in Haskell we cannot observe that at run-time. Some is not true sigma type.

From https://free-theorems.nomeata.de/
with data Proxy a = Proxy and f :: Proxy a -> b:
forall t1,t2 in TYPES, g :: t1 -> t2.
 forall t3,t4 in TYPES, h :: t3 -> t4.
  forall (x, y) in lift{Proxy}(g). h (f_{t1}_{t3} x) = f_{t2}_{t4} y

lift{Proxy}(g)
  = {(Proxy, Proxy)}

Setting t3 = t4, h = id
x :: Proxy t1
y :: Proxy t2
g :: t1 -> t2

f @t1 @t3 x = f @t2 @t4 y


Build failed

build failed
  error occurred: Command "cc" "-O0" "-ffunction-sections" "-fdata-sections" "-fPIC" "-gdwarf-4" "-fno-omit-frame-pointer" "-m64" "-DDISABLE_X64" "-Wno-unused-function" "-o" "/workspace/rust/roarningbitmap/target/debug/build/croaring-sys-7cd393670621a907/out/CRoaring/roaring.o" "-c" "CRoaring/roaring.c" with args "cc" did not execute successfully (status code exit status: 1).

Hello! Can you please run this command manually and show the output? From croaring-rs root:
cc -O0 -ffunction-sections -fdata-sections -fPIC -gdwarf-4 -fno-omit-frame-pointer -m64 -DDISABLE_X64 -Wno-unused-function -o "/workspace/rust/roarningbitmap/target/debug/build/croaring-sys-7cd393670621a907/out/CRoaring/roaring.o" -c croaring-sys/CRoaring/roaring.c

seeing the same thing in grin-wallet build, https://github.com/Homebrew/homebrew-core/pull/137857
  error: failed to run custom build command for `croaring-sys v0.4.6`

  Caused by:
    process didn't exit successfully: `/tmp/grin-wallet-20230728-9846-10uyqpj/grin-wallet-5.1.0/target/release/build/croaring-sys-7351c0e74761bdeb/build-script-build` (exit status: 101)
    --- stdout
    TARGET = Some("x86_64-unknown-linux-gnu")
    OPT_LEVEL = Some("3")
    HOST = Some("x86_64-unknown-linux-gnu")
    CC_x86_64-unknown-linux-gnu = None
    CC_x86_64_unknown_linux_gnu = None
    HOST_CC = None
    CC = Some("gcc-11")
    CFLAGS_x86_64-unknown-linux-gnu = None
    CFLAGS_x86_64_unknown_linux_gnu = None
    HOST_CFLAGS = None
    CFLAGS = None
    CRATE_CC_NO_DEFAULTS = None
    DEBUG = Some("false")
    CARGO_CFG_TARGET_FEATURE = Some("cmpxchg16b,fxsr,sse,sse2,sse3,ssse3")
    CC_x86_64-unknown-linux-gnu = None
    CC_x86_64_unknown_linux_gnu = None
    HOST_CC = None
    CC = Some("gcc-11")
    CFLAGS_x86_64-unknown-linux-gnu = None
    CFLAGS_x86_64_unknown_linux_gnu = None
    HOST_CFLAGS = None
    CFLAGS = None
    CRATE_CC_NO_DEFAULTS = None
    CARGO_CFG_TARGET_FEATURE = Some("cmpxchg16b,fxsr,sse,sse2,sse3,ssse3")
    running: "gcc-11" "-O3" "-ffunction-sections" "-fdata-sections" "-fPIC" "-m64" "-Wall" "-Wextra" "-march=native" "-o" "/tmp/grin-wallet-20230728-9846-10uyqpj/grin-wallet-5.1.0/target/release/build/croaring-sys-58c8c27055f13489/out/CRoaring/roaring.o" "-c" "CRoaring/roaring.c"
    exit status: 0
    AR_x86_64-unknown-linux-gnu = None
    AR_x86_64_unknown_linux_gnu = None
    HOST_AR = None
    AR = None
    running: ZERO_AR_DATE="1" "ar" "cq" "/tmp/grin-wallet-20230728-9846-10uyqpj/grin-wallet-5.1.0/target/release/build/croaring-sys-58c8c27055f13489/out/libroaring.a" "/tmp/grin-wallet-20230728-9846-10uyqpj/grin-wallet-5.1.0/target/release/build/croaring-sys-58c8c27055f13489/out/CRoaring/roaring.o"
    exit status: 0
    running: "ar" "s" "/tmp/grin-wallet-20230728-9846-10uyqpj/grin-wallet-5.1.0/target/release/build/croaring-sys-58c8c27055f13489/out/libroaring.a"
    exit status: 0
    cargo:rustc-link-lib=static=roaring
    cargo:rustc-link-search=native=/tmp/grin-wallet-20230728-9846-10uyqpj/grin-wallet-5.1.0/target/release/build/croaring-sys-58c8c27055f13489/out

    --- stderr
    thread 'main' panicked at 'Non floating-type complex? Type(_Complex _Float16, kind: Complex, cconv: 100, decl: Cursor( kind: NoDeclFound, loc: builtin definitions, usr: None), canon: Cursor( kind: NoDeclFound, loc: builtin definitions, usr: None)), Type(_Float16, kind: Float16, cconv: 100, decl: Cursor( kind: NoDeclFound, loc: builtin definitions, usr: None), canon: Cursor( kind: NoDeclFound, loc: builtin definitions, usr: None))', /github/home/.cache/Homebrew/cargo_cache/registry/src/index.crates.io-6f17d22bba15001f/bindgen-0.56.0/src/ir/context.rs:1957:26
    note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
  warning: build failed, waiting for other jobs to finish...
  error: failed to compile `grin_wallet v5.1.0 (/tmp/grin-wallet-20230728-9846-10uyqpj/grin-wallet-5.1.0)`, intermediate artifacts can be found at `/tmp/grin-wallet-20230728-9846-10uyqpj/grin-wallet-5.1.0/target`

let me know if I need to provide more details.

Does this also happen with the latest version of croaring-rs?

These appear to be two separate issues:
@takakawa's issue
This appears to be an error to compile the C source code. I don't have much info, though I will say the combination "-m64" "-DDISABLE_X64" looks suspicious to me, but I don't see anything that uses a define called that, I see one called ROARING_DISABLE_X64. CRoaring's build script doesn't look like it would ever add a flag like that, so I'm guessing it came from e.g. CFLAGS.
@chenrui333's issue
This appears to be https://github.com/rust-lang/rust-bindgen/issues/2500. An easy workaround would be to use the pre-built bindgen option (don't enable the buildtime_bindgen feature)

This appears to be rust-lang/rust-bindgen#2500. An easy workaround would be to use the pre-built bindgen option (don't enable the buildtime_bindgen feature)

that might be difficult for us to do.

For a while now, the buildtime_bindgen feature has just been a noop: croaring-rs unconditionally uses pre-generated bindings.


Minor Fixes

Verify User Is Able to Activate Anaconda Professional.
The kw which is checking the KfDef takes only the first KfDef instead of looping over all of them. Since in v1.15 Anaconda has its own KfDef, the kw is not able to capture it.

Verify Intel AIKIT Operator Can Be Installed Using OpenShift Console.
Increased the time for checking if the service is available in the dashboard

Verify User Is Able to Produce and Consume Events (RHOSAK)
RHODS Dashboard changed Window title in the last release, so this PR changes it in the TC

Keyword "Remove Disabled Application From Enabled Page"
screenshot is replaced everytime by the last TC run because it uses static name. PR changes it by appending the ISV app id in the screenshot title

ods-ci.pod.yaml
Syntax error fixed

test case "Verify That Idle JupyterLab Servers Are Culled In Sandbox Environment After 24h"
RHODS Dashboard changed Window title in the last release, so this PR changes it in the TC

test case "Verify RHODS Dashboard Explore And Enabled Page Has No Message With No Component Found". A config map has changed title breaking the TC. The title will be changed back but for the moment I will patch the TC to work with the new name (bug has been raised)

Signed-off-by: bdattoma<EMAIL_ADDRESS>

fixes 1),2),4) still in progress due to some failures obtained in previous Jenkins runs
fix 3) verified with Job#1184 SUCCESS
fix 5) tested offline (not possible to test with Jenkins at the moment)
fix 6) not feasible the testing for this PR, however it very simple change in the window title (same as 3 which is verified)
fix 7) is "passed" in the Job#1184 but it is expected to fail, unless the bug has been resolved. Need investigation


Adding an import with quick fix should respect sort order

TypeScript Version:  2.3
Code
Before the suggested B import is added:
import { A, C } from "./letters";

const b: B = new B(); // suggestion to import B from "./letters";

After:
import { A, C, B } from "./letters"; // TSLint's ordered-import rule is now offering a quick fix

const b: B = new B();

Expected behavior:
If the imports are sorted, adding a new one should add in sorted order. In this case it would be { A, B, C }.
Actual behavior:
The added import is tacked onto the end.
If the preference for keeping imports sorted will be added in #10020, it seems like it would make sense to respect that order in individual adds as well.

IME this has since been resolved. Thanks!


Locations not allowing for continuation of Quests.

What is your Quests version / build number (do not say "latest")?
4.0.3 Updated server 24/05/2021
Describe the problem and how to replicate it. Any console errors to post using a pastebin?
After updating to version listed above previously working location data stopped working.
Step 1; Create Quest with Location Data
Step 2; Save and reload questadmin to apply new Quest
Step3: Accept quest from NPC and goto location; No Console error occurs.
Step 4: unable to continue to Step 2 of Quest.
Steps are repeated as such on a fresh Spigot server with no other mods.
If applicable, can you provide an example quest from quests.yml using a pastebin?
https://pastebin.com/iAZxm6Hy

Thanks for the report. Which version did you update from?

Thank you for getting back to me, the version that i updated from was 4.0.2, I tend to update quite regularly.

Probably related to #1688 will look into it.

Same issue in 4.0.3 fresh download

I apologise for the double post In that case @PikaMug
May I note, it happens like @JustATin555 states, on a fresh download of 4.0.3 without any other mods than the basic spigot installation.


Appstudio update hacbs-test-project-ih1o

Pipelines as Code configuration proposal
To start the PipelineRun, add a new comment with content /ok-to-test
For more detailed information about running a PipelineRun, please refer to Pipelines as Code documentation Running the PipelineRun
To customize the proposed PipelineRuns after merge, please refer to Build Pipeline customization

Pipelines as Code CI has failed.There was an error creating the PipelineRun: hacbs-test-project-ih1o-on-pull-request-
creating pipelinerun hacbs-test-project-ih1o-on-pull-request- in namespace rhtap-demo-rfxr-tenant has failed.
Tekton Controller has reported this error: the namespace of the provided object does not match the namespace sent on the request


fixed duplicate event binding in devices which supports both pointer …

Issue Reference:
On device which supports pointer events and touch events, the two event listener are getting bind, which is triggering event twice one event with pointer and another touch.
How to Reproduce:
Run Ractive component with on-tap event attached in latest chrome browser (which have window.PointerEvent object) with mobile view simulator, once click on component with on-tap event the event listener executed twice.
Solution:
Fixed duplicate event binding in devices which supports both pointer and touch event, skipping touch event binding in case device supports pointer event.

I'm not sure how this fixes the problem, there are still 2 event listeners.

I'm not sure how this fixes the problem, there are still 2 event listeners.

I'm not sure how this fixes the problem, there are still 2 event listeners.

Hi Ivan, can you please provide more information which 2 event listeners you are pointing out, it is a mouseDown and touchStart events, then though there are two event listener the supporting one will only get fired based on the device either mouseDown or touch but not both.

I'm not sure how this fixes the problem, there are still 2 event listeners.

Hi Ivan, can you please provide more information which 2 event listeners you are pointing out, it is a mouseDown and touchStart events, then though there are two event listener the supporting one will only get fired based on the device either mouseDown or touch but not both.


Add SimpleCell.

Fixes #3681

Something went awry.

Codecov Report
Attention: 12 lines in your changes are missing coverage. Please review.

Comparison is base (6181239) 59.02% compared to head (6be118c) 59.10%.
Report is 10 commits behind head on main.

Files
Patch %
Lines

flax/linen/recurrent.py
57.14%
12 Missing :warning:

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3697      +/-   ##
==========================================
+ Coverage   59.02%   59.10%   +0.07%
==========================================
  Files         103      103
  Lines       12391    12454      +63
==========================================
+ Hits         7314     7361      +47
- Misses       5077     5093      +16

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Binding Attributes with Some Static Value

I want to be able to create an anchor tag inside of a rv-each. Here is the code that doesn't work.

    {project.id}

{project.title}

How do I go about binding an href like this?

I ended up making something a bit more flexible. It's a way to either combine multiple properties together or combine them into some string.  Here's how it looks in the markup:
My Link

When you use the formatter in the markup, you first pass the string you want to output. Then you pass any additional values you want to use in the string besides the initial value that's specified before "pipe" character.
In my example, the string replacement looks at:
@0 is the initial property "group.Id"
@1 is the "group.Name"
@2 is the "group.Owner"
The formatter will accept as many arguments as you want.
Here's the js to add this formatter to rivets (using a jQuery $.each)
rivets.formatters.text = function (value, text) {
    text = text.replace('@0', value);

    var args = (arguments.length === 1 ? [arguments[0]] : Array.apply(null, arguments));
    args.shift(); args.shift(); //remove the first two arguments from the array

    $.each(args, function (i) {
        var replaceChar = '@' + (i + 1).toString();
        text = text.replace(replaceChar, this);
    });

    return text;
};

Check out Rivets StdLib for another implementation (easily converted to TinyBind)


how i can track velocity

how i can track velocity only vertically

"Velocity"
Vector Velocity = localPlayer->velocity();
"Velocity Horizontal"
float Speed = Velocity.length2D()
"Velocity Vertical"
float Speed = Velocity.z;


Changed Created UAN Boot Images for CASMUSER-3059

Summary and Scope
Changed the text and command in one step and inserted another step for a WAR.
docs-only change made necessary by SKERN-6718. This PR is for UAN 2.4
Issues and Related PRs
CASMUSER-3059
SKERN-6718
Testing
Local docs builds made on WSL2/Ubuntu and formatting verified on the test ITG docs portal.
Risks and Mitigations
No known risks.
Pull Request Checklist

[ ] Version number(s) incremented, if applicable
[ ] Copyrights updated
[*] License file intact
[*] Target branch correct
[ ] CHANGELOG.md updated
[*] Testing is appropriate and complete, if applicable
[ ] HPC Product Announcement prepared, if applicable

Declining this one and recreating to target UAN 2.4

https://github.com/Cray-HPE/uan-product-stream/pull/43


Convert FileBrowser to TS/Composition API

Also merges the PublishFileBrowser component into FileBrowser.vue as part of #903.
I also changed the CI config so tests will run on any branch, not just master or branches that have PRs associated with them; GitHub Actions minutes are unlimited for public repositories so I don't see why not.

@dchiquito this should be all set for you to review again, PTAL. FYI to save you some time, the only code that changed since your last review is the assetPaths() return type in rest.ts and the contents of the <script> section of FileBrowser.vue. Mentioning this b/c I removed a redundant top-level <div> in that component and now the diff view is showing the entire template as new code.


[KOA-6262]: Updated explore icon

Updates the explore icon
Remember to include the following changes:

[x] UNRELEASED.md (See How to write a good changelog entry)
[ ] README.md
[x] Tests
[ ] Storybook examples created/updated for changes to tokens and icons

@olliecurtis the CI is failing, seems like something went wrong with the build icons script? 🤔


Removed Hytch, Spire Africa, and Zumi

I removed Hytch, Spire Africa, and Zumi as these companies are no longer operating.

@ezinneanne Thank you for helping us keep the tech companies web app updated.


LiteX#master compatibility

We currently don't support LiteX master (as of 2022-08-24), with dma_test resulting in an instantaneous reboot, even without a kernel panic (i.e. no crashdump). The 2022-04 tag works, as does the following set of slightly more recent subproject hashes:
amaranth,39a83f4d995d16364cc9b99da646ff8db6394166
litedram,f39625372901a5fbbfb4311e2e08c096a919ec7b
liteeth,d10cda84b6367129f7d839b1f9b90d3b4522f45e
litehyperbus,a950fa768d255869a36d8c91d7bd4dbd3c47a9b2
liteiclink,f3eadbd1165a8055c7c2c98b72c450ca1bdcc09f
litejesd204b,0c90b3ebd0ce4ec0ad66fae974f376daaff86ced
litepcie,18985c79ca7be1e05b00ec2bbe61372a59fd4221
litesata,a6da1885d5b5126ce0531d9b6182712bf63e8d33
litescope,d1c480d0e8c1e6188cccab4a0621c714376d591c
litesdcard,809ade33137edd6db59912844b4a7ccfbe2e4e0c
litespi,7d32810ee05eb96e5cf4152483a3e1e501f9113c
litex-boards,74db821f672e3986f4408957002e88cf55ac65ad
litex,3959ca39d46700865c825051b333296233d03d29
litex-boards,74db821f672e3986f4408957002e88cf55ac65ad
migen,d4e3f34177c32f09904397179e6ed9c83175e528
pythondata-cpu-blackparrot,ba50883f12d33e1d834640640c84ddc9329bb68a
pythondata-cpu-cv32e40p,5cb0e3dcbb6bb2e80cfaceeeab553b21b1be3921
pythondata-cpu-ibex,d3d53df613c2878152eaa382a8cd558efdf21ec6
pythondata-cpu-lm32,cfd9dabc880857afd22998e1cf24d545fe33a03a
pythondata-cpu-microwatt,dad611c3e0a39c520d94ce101a6aad6112ac9ecf
pythondata-cpu-minerva,64b32cd6f86b959a2c15db1c8244596e6e1b9438
pythondata-cpu-mor1kx,6c7360c17294e3ca3788f61fa3212443464fe2dc
pythondata-cpu-naxriscv,2f6fd32e62d5196b412c6aac92136165af2948a4
pythondata-cpu-picorv32,c1b9e6ae9fb231f318f764665733a9fa4700ea74
pythondata-cpu-rocket,0821742eee59da32491d8b7ffc550ac7c056fb9e
pythondata-cpu-serv,faef269e58d32c43931ec88878d89830b5eb84fd
pythondata-cpu-vexriscv,cdfb2306087a680df65153760fd41002e316ad30
pythondata-cpu-vexriscv-smp,60c32b141906ea4b8ba6ec19c66a321a1477e55c
pythondata-cpu-vexriscv-smp,60c32b141906ea4b8ba6ec19c66a321a1477e55c
pythondata-misc-tapcfg,05ce0fc2954a109896f8d79dedc39d29caa39fd7
pythondata-misc-usb_ohci,1f9c77314d3b91fe3194245c3e3197a075803199
pythondata-software-compiler_rt,fcb03245613ccf3079cc833a701f13d0beaae09d
pythondata-software-picolibc,ae9c1ca4af911c85f941c8a8540ddae4520ae2cd

The issue seems to be in litepcie_dma_reader_start, where setting the DMA table entries breaks the system after a couple of iterations, so I guess it makes the device write to arbitrary memory. I originally thought the 64bit changes from #1 might be related, but checking-out everything right before that didn't help.

After chatting with Florent, this is likely due to the DMA interface change that was necessary to support 64-bit mode in LiteX. Specifically, these changes are probably required:

https://github.com/enjoy-digital/litepcie/commit/ea6e933f2526a48ec48c02edaac45ed5cf3c79fd#diff-b58dc917d1602cb671dfdb504a9919722f9b992b1c1760924ae1dd8acc0f2010L198-R204
https://github.com/enjoy-digital/litepcie/commit/ea6e933f2526a48ec48c02edaac45ed5cf3c79fd#diff-b58dc917d1602cb671dfdb504a9919722f9b992b1c1760924ae1dd8acc0f2010L252-R258

The change was made to unify 32 and 64 bit mode:

write 32 LSB to PCIE_DMA_READER/WRITER_TABLE_VALUE_OFFSET + 4
if applicable, write 32 MSB to PCIE_DMA_READER/WRITER_TABLE_WE_OFFSET
finalize the descriptor by writing to PCIE_DMA_READER/WRITER_TABLE_WE_OFFSET


[PatchRelease] Prepare 2024 August Patch Release

API change check
API changes are not detected in this pull request.


Gateway Manager Port

Hello,
The "Important" note says the following:
"Enable port 4443 for inbound traffic from Gateway Manager for session monitoring to work."
In the documentation on working with NSGs, it only mentions 443. Is this a typo? Here's the exact wording on that page:
"Ingress Traffic from Azure Bastion control plane: For control plane connectivity, enable port 443 inbound from GatewayManager service tag. This enables the control plane, that is, Gateway Manager to be able to talk to Azure Bastion."

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 2d8f8711-29cb-4d2a-54f2-4be097c4b319
Version Independent ID: cf1d18e3-5d83-b982-29a4-4a95a10f78a0
Content: Azure Bastion session monitoring and management
Content Source: articles/bastion/session-monitoring.md
Service: bastion
GitHub Login: @cherylmc
Microsoft Alias: cherylmc

@jons-own-stuff  , Thank you for your feedback. We will investigate and get back to you shortly.

@jons-own-stuff , yes 4443 is used for the communication between Gateway Manager and the Bastion for the session monitoring to work.
We have plans to change that to 443 in the future. But as of now, just add a inbound rule to allow 4443 from Gateway Manager Tag


use sscanf_s in MSVC

Without this, MSVC complains about sscanf being unsafe.

Coverage remained the same at 50.377% when pulling b40e8c942764e4923d6ef31767ea2949b8a1e22b on nlguillemot:master into ad9911ef1b124c628581c0f75adb3178f9d03c3c on syoyo:master.

Cool! Thanks!


Zephyr v3.7.1

boards: disco_l475_iot1: fix arduino_i2c config
arch: riscv: stacktrace: fix user thread stack bound check
soc: s32k3: fix RAM retention
net: sockets: tls: Fix poll update event check
net: wifi: Fix the nm type check error
net: ptp: Adjust clock using correct offset
net: ptp: Properly handle second overflow/underflow
net: net_if: fix net_if_send_data for offloaded ifaces
drivers: adc: fix missing ref_internal in adc_sam0
net: coap: Fix underlying type for block number
net: coap: Parse more flag in coap_get_block2_option()
doc: migration-guide-3.7: Add entry about CoAP block API change
drivers: bluetooth: hci: Fix stm32wb BLE behavior
drivers: bluetooth: hci: Fix stm32wb BLE behavior
tests: device: test CONFIG_DEVICE_DT_METADATA=y
device: remove const qualifier from node label copy
samples/shields: x-nucleo-iks4a1: Add lsm6dsv16x temp display
sensor: lis2dux12: fix SENSOR_ATTR_FULL_SCALE case
drivers/sensor: lis2dux12: fix odr and range
drivers/sensor: lis2dux12: fix temperature conversion
drivers: i3c: common: Do not tread GETCAPS failure as error for 1.0 device
dts: fix warnings in nxp_rt11xx.dtsi
drivers: usb: stm32 udc driver get the global otg interrupt
boards: st: sensortile_box_pro: remove hts221 sensor
icm42688: Fix divide by zero potential
drivers: mdio: Clear interrupt signal in ISR
bluetooth: avoid USB starvation in bap_broadcast_sink
Revert "net: socket: fix sys/_timespec.h not found error"
net + posix: avoid host includes when building for native_sim
posix: sys/stat.h: move declarations to posix_types.h
posix: posix_types.h: simplify conditions on pthread types
tests: posix: net: open native_sim in c++,posix,net,native case
net: tcp: Keep track of recv window size change since last ACK
bl5340_dvk: Move MIPI_DBI interface back to spi4
samples: tfm_integration: psa_crypto: Disable sample
Bluetooth: Host: Fix compiling PAwR Sync without PAST RX
net: context: Check null pointer in V6ONLY getsockopt
arm: debug: gdbstub: fix header
tests: gdbstub: minor refactor to the testcase.yaml
tests: gdbstub: add build-only test
logging: Trigger logging thread when we start to drop messages
drivers/sensor: lis2dux12: fix mode when setting odr
linker: esp32: fix cpp rom region
Bluetooth: host: add more info to conn.c log
Bluetooth: allow compiling host with CONFIG_NO_RUNTIME_CHECKS
Bluetooth: L2CAP: Mark user_data as owned by the stack
Bluetooth: adapt l2cap/userdata test
Bluetooth: remove forgotten TODO
Bluetooth: L2CAP: Set NULL callback for PDUs
manifest: update percepio
Bluetooth: Controller: Fix in-system ISR profiling for adv and conn
Bluetooth: Controller: Use uint16_t to store ISR profiling value
Bluetooth: Controller: Add ISR profiling using ticker ticks
Bluetooth: Controller: Verbose radio is ready assertion
Bluetooth: Controller: Fix ISR profiling for single timer use
Bluetooth: Controller: Fix empty PDU buffer overrun under ISR latency
Bluetooth: Controller: Fix PDU length in case of ISR latency issue
Bluetooth: Controller: nRF51x: Fix regression in encrypted connection
Bluetooth: Controller: Fix regression using speed optimization
Bluetooth: Controller: Relax radio packet pointer assignment deadline
Bluetooth: drivers: userchan: Fix gcc 13 overflow warning
drivers: modem: cellular: hl7800 fix enter cmux
dts: arm: nxp: lpc55sxx: fix sram node address
net: lib: http_server: Clear http1_headers_sent flag on new request
soc: arm: nxp lpc55xx: fix nxp,ctimer-pwm init procedure (attach clock)
Bluetooth: Host: Free ISO TX context
Bluetooth: Host: Unref ISO fragments after disconnection
Tests: Bluetooth: Add another ISO frag test
fs: fuse: ensure S_IFxxx macros are available
Bluetooth: L2CAP: Add re-assembly stress test
Bluetooth: hci_common: Add assert on buf allocation
Bluetooth: Host: Free ACL RX fragments on disconnection
entropy_smartbond: Clear pending interrupts after disabling TRNG
gpio_smartbond: Set pin to input when it is configured as GPIO_DISCONNECTED
gpio_smartbond: Remove pdc entry only if it exists
gpio_smartbond: Fix port_set_masked_raw
Bluetooth: Host: Add disconnection during TX fragmentation test
Bluetooth: Host: Add missing bt_tx_irq_raise()
soc: rt11xx: Fix dual core ENET PLL
soc: rt11xx: Fix bus clocking
net: websocket: fix undefined reference
dts: nxp_rt1010: mark edma channel has separate interrupt entry
ci: rerun issue check on PR edit
drivers/sensor/st: Fix wrong data byte swap for be
drivers: sdhc: imx_usdhc: assume card is present if no detection method
Bluetooth: Host: Fix incorrect build assert
tests: Bluetooth: disable optimizations in unit test
net: sockets: Update msg_controllen in recvmsg properly
tests: net: socket: udp: clear control data buffer before recvmsg
modules: percepio: Fix python executable name
bluetooth: host: adv: Release buf if failed to set addr
drivers: i2s: i2s_ll_stm32: Handle single clock source
linker: section_tags: fix missing include
drivers: ethernet: nxp_enet: Fused MAC address fixes
manifest: mbedtls: update to 3.6.1
doc: release notes: add 3.7.1
samples/llext/modules: build hello_world as built-in by default
samples/llext/modules: sample.yaml fixes
serial: uart_async_rx: Avoid possible division by zero
bugfix: soc: esp32: disable interrupts in flash operation
soc: arm: nxp: fix USB w/ SPEED_OPTIMIZATIONS
devicetree: add DT_INST_NUM_IRQS()
arch: riscv: fill all IRQ stacks with 0xAA
net: dns: Add per socket user data for the dispatcher
net: mdns_responder: Set socket of service for statistics purposes
dts: arm: st: stm32h5: fix spi 1-3 clocks
net: l2: ieee802154: fix deadlock
net: if: Check chksum offloading properly for VLAN interfaces
scripts: Remove wrong bz package
net: l2: ieee802154: fix typo
net: l2: ieee802154: move vars to top of block
net: l2: ieee802154: decouple frame decryption from upper layer fields
net: sockets: tls: Prevent infinite block during handshake
drivers: intc: plic: fix IRQ on every hart regardless of mapping
bluetooth: AVDTP: Check buffer len before pulling data
usb: fix wrong capacity report for USB mass storage device
arch: riscv: isr.S: restore s0 before jumping to z_riscv_fatal_error_csf
tests: arch: riscv: test csf registers value
net: fix handle unaligned memory access in net_context_bind()
cmake: Require Python >= 3.10
soc: esp32s3: fix initialization code
doc: services: device_mgmt: mcumgr: Correct license for tool
drivers: cc: mcux: Fix incorrect clock source of FlexSPI2
west: blobs: prefer constants to literals for blob status
west: blobs: verify fetched blobs after downloading
ethernet: phy_mc_ksz8081: Don't reset in cfg link
sensor: tsl2591: fix: Address CID 353644 & 353654
modules: mbedtls: Enable PEM writing when PEM cert format is selected
arch: riscv: stacktrace: fix output without ra on the stack top
tests: arch: common: stack_unwind: add qemu_riscv32e
Bluetooth: L2CAP: document memset requirement
modbus: reset wait semaphore before tx
sysbuild: cmake: Fix ExternalZephyrProject_Add() revision handling
doc: fix issue with keydown/keyup being ignored
boards: hardkernel: odroid_go: fix PSRAM size
soc: nxp: lpc55xxx: fix dependencies for SOC_FLASH_MCUX
drivers: udc: add opaque pointer to store upper layer private data
manifest: esp32: update hal_espressif
sample: input_dump: enables INPUT_ESP32_TOUCH_SENSOR for Espressif boards
net: ip: igmp: Fix wrong header length
net: ip: igmp: Remove too strict length check
net: ip: igmp: Add igmp.h for definitions
test: net: igmp: Add extra IGMPv3 testcase
drivers: dai: sai: don't crash on underrun/overrun
irq: multilevel: fix irq_parent_level_3()
drivers: espi: host_subs: npcx: fix LDN register definition of PMCH
mbedtls: update to 3.6.2
drivers: rtc: rv3028: preserve CLKOUT when enabling callbacks
cmake: support range for find_package(Zephyr-sdk)
cmake: limit Zephyr SDK to 0.16.x series for Zephyr 3.7 LTS
SDK_VERSION: Use Zephyr SDK 0.16.9
ci: build samples/cpp/hello_world as part of the multiplatform test
doc: css: fix styling of signatures
testsuite: coverage: fix typo in the CMakeLists.txt
xtensa: mpu: make sure write to MPU regions is atomic
xtensa: mpu: update hardware if manipulating current domain
manifest: Update EDTT to latest version
net: ipv4: Fix ARP probe check in address conflict detection
drivers: flash: flash_mcux_flexspi: add support for W25Q512JV
boards: nxp: frdm_rw612: correct max frequency for WS25Q512JV
mgmt: mcumgr: stat: Fix stat_mgmt_list behavior
boards: Fix to accommodate for slowly tests on UP2
drivers: flash: spi_nor: fix config struct
drivers: flash: spi_nor: simplify defines
drivers: can: sam: fix sys_write32() arguments order
samples: smp_svr: fix overlay-udp.conf build
drivers: display: stm32_ltdc: fix return value
soc: nxp: lpc55s69: Fix part number typo
shell: devmem: minor optimizations
shell: shell_help: fix width for subcommands help text
tests: dma/chan_blen_transfer: relocate TX and RX buffers
tests: dma/chan_blen_transfer: move buffer to DMA-accessible memory
Revert "arch: arm: cortex_m: restore comment lost in translation"
Revert "arch: arm: cortex_m: move part of swap_helper to C"
arm: cortex_m: restore fix for loading z_arm_int_exit
storage/stream_flash: Fix range check in stream_flash_erase_page
tests/storage/stream_flash: Add erase range check test
mgmt: smp: shell: Respond on uart shell device when changed at runtime
drivers: mdio_nxp_enet: Don't disable IRQ
drivers: ethernet: ksz8081: Fix reset times
drivers: ethernet: ksz8081: RMII override RNB part
drivers: nxp_enet: Fix tx error handling
doc: fix body length in SMP UART transport
kernel: fatal: Fix NO_OPTIMIZATIONS build
doc: releases: update release notes for 3.7.1
release: bump version to 3.7.1-rc1
arch: riscv: add macro to access hardware registers
arch: riscv: reset global pointer on exception
tests: arch: riscv: make sure that gp reg can't be corrupted
tests: thread_error_case: no ICOUNT for qemu/xtensa/dc233c/mmu
xtensa: remove mem_domain excess padding
arch: arm: cortex_m: move _main in input list
arch: arm: cortex_m: add memory to the clobber list
driver/dma_smartbond: Fix driver initialization when PM_DEVICE is set
drivers: counter: Fix unbalanced policy state lock
manifest: Update TF-M
west.yml: MCUboot synchronization from v3.7-branch
doc: release: 3.7: Add MCUboot/TF-M updates for 3.7.1
[v3.7-branch] west.yml: fix phy nested critical section
scripts: twister: Fix serial_py referenced before assignment
bluetooth: gatt: Fix ATT Read By Type by DB change unaware client
doc: release notes: update with new issues resolved
release: Bump version to 3.7.1

oops


Desktop Using Trained Pipelines

Pipeline outputs follow the way the web is done with a {type}.pipe file and a .zip file that are moved into a folder in the DIVE_Pipelines.  This folder is spread onto the pipeline object with the category of 'trained' when the pipeline list is requested.

After a pipeline is complete it will check the job folder for a category_models folder and then check the contents for the [type].pipe file.  Then it will move that into the DIVE_Pipelines directory.
Modified the exit function for training to include some errors if it can't find the .pipe file or directory.  I wish I knew a bit more about what is expected in there for the different training configurations.

Question:

Let me know if you think it should be a JSON file indexed with trained pipeline folder instead of utilizing the folder names.  The one nice thing about folder names is that if a user wants to import or delete pipelines right now they can do it by going to the folder and either adding or removing the folder.

Note:
This has had the basic train_netharn_cascade.viame_csv.conf testing process done on it.
Future PRs:

Deletion of trained pipelines through an interface.  I don't know if this lives in "Training" or somewhere else but it may be nice to do that.
As an aside to the above I think I want to implement dataset deletion from the recents folder
Importing trained pipelines from the web to the desktop version (I've tested this manually by moving the files into a proper folder and it works well at least for train_netharn_cascade.viame_csv.conf pipelines).  This could be done in a few ways:

I think either having them provide a folder and giving it a name and it moves/copies the items to that directory after some basic checking to make sure there is a .pipe and other data
Allowing a zip file to be uploaded and we use something like jszip or other utilities to check it.
Advanced?? - we provide them with a link directly to the folder and let them add/remove files.  Obviously this has the potential for allowing to break stuff as well.

Looks like u got some conflicts.


Following naming standards

All-uppercase letters stand out and make the file easily visible which makes sense because it is probably the first thing a new user would want to look at. (Or, at least, should have looked at) Also file names starting with a capital letter will be listed before lower-case names in ASCII alphabetical sorting (LC_COLLATE=C) which helps make the file visible at a first glance.
Here is the more details about it : http://softwareengineering.stackexchange.com/questions/301691/readme-txt-vs-readme-txt
Also tell future contributors to follow this style. :)

Hey @manishbisht
Please squash the commits & fix the conflicts 👍