Federator cannot create ConfigMaps due to registration namespace resourcequotas

Cluster Setup

Kubernetes version:
Type of Cluster (RKE1, RKE2, k3s, EKS, GKE, AKS, OKE, etc): RKE1, imported
Installation option (Helm Chart / Custom Installation):

If Helm Chart, did you supply any non-default values? If so, what values did you supply:
If Custom Installation, how did you install it (kustomize, k8s manifest, etc.):

Describe the bug
The automatically created namespace cattle-project-p-xxxxx uses a projects default namespace resource quotas. If a project already has fully-booked project resource quotas, this creates a namespace with a ResourceQuota that allows no objects to be created in it:
apiVersion: v1
kind: ResourceQuota
metadata:
  creationTimestamp: "2023-03-09T10:36:52Z"
  generateName: default-
  labels:
    cattle.io/creator: norman
    resourcequota.management.cattle.io/default-resource-quota: "true"
  name: default-2dfbc
  namespace: cattle-project-p-5tmh6
spec:
  hard:
    configmaps: "0"
    limits.cpu: "0"
    limits.memory: "0"
    persistentvolumeclaims: "0"
    pods: "0"
    replicationcontrollers: "0"
    requests.cpu: "0"
    requests.memory: "0"
    requests.storage: "0"
    secrets: "0"
    services: "0"
    services.loadbalancers: "0"
    services.nodeports: "0"
status:
  used:
    configmaps: "2"
    limits.cpu: "0"
    limits.memory: "0"
    persistentvolumeclaims: "0"
    pods: "0"
    replicationcontrollers: "0"
    requests.cpu: "0"
    requests.memory: "0"
    requests.storage: "0"
    secrets: "1"
    services: "0"
    services.loadbalancers: "0"
    services.nodeports: "0"

This results in the federator throwing the following error:
time="2023-03-09T10:43:16Z" level=error msg="error syncing 'p-5tmh6': failed to create cattle-project-p-j7dqd/monitoring.cattle.io.v1alpha1 /v1, Kind=ConfigMap for cattle-project-p-5tmh6-monitoring-data cattle-project-p-5tmh6: configmaps \"monitoring.cattle.io.v1alpha1\" is forbidden: exceeded quota: default-2dfbc, requested: configmaps=1, used: configmaps=1, limited: configmaps=0, requeuing"

To Reproduce

Create a project with resource quotas
Create a namespace that fully uses the quotas
Enable project monitoring for the project

Result
The aforementioned problem appears in the logs. The configmap containing the helm chart values isn't saved in the user's registration namespace.
Expected Result

The config map should be present (along with everything else the federator creates) and no error should appear in the federator's logs.
Screenshots

None needed.
Additional context

Please consider not using the default namespace resource quotas when creating the registration namespace. They may be high, which creates a namespace that unnecessarily binds precious resources from the project, like CPU and RAM. Consider using dedicated ResourceQuotas for the creation of this namespace.

SURE-6035


explorer: update to version 0.3.2

Update instructions to install v0.3.2 with NodeJS 14

Hello,
I did not know where to put this so I will just write it down here.
With an eye towards the upcoming V3 of this guide and switch to cookie file instead of rpc user and password authentication there might be an issue here.
In the .env file there is an option to set the cookie file
#BTCEXP_BITCOIND_COOKIE=/path/to/bitcoind/.cookie
however user btcrpcexplorer can not access this file because it has mode 600 and belongs to the bitcoin user.
There is a workaround running the bitcoin service with umask 0027 and add user btcexplorer to the bitcoin group which I found here

In the service unit add a UMask=0027 line under [Service]. Also add sysperms=1 to your conf file. Now when started the default system perms are 640 and the cookie is created as desired. The only thing is other files have same perms (like mempool.dat and peers.dat). I'm not too worried about that as they are read only for the bitcoin group and still blocked to others.

I do not know if the above is a good idea security wise and maybe it is better to run btcexplorer from user bitcoin direclty?

switch to cookie file instead of rpc user and password authentication there might be an issue here.

I'm aware of that. We'll probably use the "RPC auth" method that allows to use rpcuser/pw + cookie authentication in parallel. Adding sysparms=1 to the bitcoin.conf gives the "bitcoin" group read-only access to the cookie, but this only works with the wallet in Bitcoin Core disabled.
Thanks for pointing this out. I think for this update here this should not yet be relevant.

This pull request was resolved with the update to RaspiBolt version 3.


scsi_volume_id_device_path_resolver flaky on ubuntu jammy

We've gotten reports of intermittent agent_timed outs for deployments on vsphere with jammy stemcells (though it should also apply to bionic stemcells because it is caused by a change in the kernel)
These we're caused by the agent failing to setup the ephemeral disk attached to the VM because the resolver code never entered the if statement: if path.Base(rootDevicePath) == "sda"
Investigation on the VM showed that:
ls /sys/bus/scsi/devices/32:0:0:0/block => sdb
ls /sys/bus/scsi/devices/32:0:1:0/block => sda
Disk /dev/sda: 16 GiB,<PHONE_NUMBER>4 bytes, 33554432 sectors
Disk model: Virtual disk
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk /dev/sdb: 5 GiB,<PHONE_NUMBER> bytes, 10485760 sectors
Disk model: Virtual disk
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x54a191e8

Device     Boot Start     End Sectors  Size Id Type
/dev/sdb1          63 9998046 9997984  4.8G 83 Linux

The good news is that the VolumeID the CPI  creates as a disk hint, still helps identifying the right /dev/sdX path for the ephemeral disk.
https://github.com/cloudfoundry/bosh-vsphere-cpi-release/blob/master/src/vsphere_cpi/lib/cloud/vsphere/cloud.rb#LL694C11-L694C56
We relied on the root disk to always be exposed at /dev/sda. Which has worked for ages (the code essentially didn't change since 2014) but actually is now problematic since it assumes that naming of /dev/sd* is deterministic.
Which according to
https://www.suse.com/support/kb/doc/?id=000018449
was the case until kernel version 5.3
The reliance seems unnecessary though, since both disks (os-root from the stemcell and the ephemeral disk) are associated  with the same scsi host (and the agent already assumes they are on they same host. So finding the hostID should be safe with either /dev/sdX).
fixes:
[linuxPlatform] 2022/12/02 13:56:04 INFO - Setting up raw ephemeral disks
[File System] 2022/12/02 13:56:04 DEBUG - Glob '/sys/bus/scsi/devices/*:0:0:0/block/*'
[linuxPlatform] 2022/12/02 13:56:04 DEBUG - Error getting ephermeral disk path Zero length hostID
[linuxPlatform] 2022/12/02 13:56:04 INFO - Setting up ephemeral disk...
[File System] 2022/12/02 13:56:04 DEBUG - Glob '/var/vcap/data/*'
[File System] 2022/12/02 13:56:04 DEBUG - Making dir /var/vcap/data with perm 0750
[main] 2022/12/02 13:56:04 ERROR - App setup Running bootstrap: Setting up ephemeral disk: No ephemeral disk found, cannot use root partition as ephemeral disk
[main] 2022/12/02 13:56:04 ERROR - Agent exited with error: Running bootstrap: Setting up ephemeral disk: No ephemeral disk found, cannot use root partition as ephemeral disk
[arping] 2022/12/02 13:56:04 DEBUG - Broadcasting MAC addresses
[File System] 2022/12/02 13:56:04 DEBUG - Checking if file exists /sys/class/net/eth0
[main] 2022/12/02 13:56:04 DEBUG - Starting agent
[File System] 2022/12/02 13:56:04 DEBUG - Reading file /var/vcap/bosh/agent.json
[File System] 2022/12/02 13:56:04 DEBUG - Read content

Thanks @nouseforaname


2020 COMMIT

Review the changes

Thank you for your pull request. I merged it successfully and created release 2<IP_ADDRESS>. I was still not able to compile for Revit 2020 right out of the box, though. I updated the Revit API .NET assembly paths again for Revit 2020, cf. HintPath settings, was able to compile then and created release 2<IP_ADDRESS>.


Can not install on TFS

Hi
Is there a reason why It is not possible to install this Drop plan on TFS 2017 update 2?
The install button is gray.
Regards
Gitte

Hi
It should be possible, i see successful installations on that version.


⚠ Bofrost.lu has degraded performance

In d46e59c, Bofrost.lu (https://www.bofrost.lu) experienced degraded performance:

HTTP code: 200
Response time: 1451 ms

Resolved: Bofrost.lu performance has improved in 8a13846 after 10 minutes.


Questions about learning rate, betas, and weight decay in Adam optimizer

I am wondering why learning rate is set to 0.0002 while your paper did mention that We use Adam [25] optimizer
with a constant learning rate of 0.0001 to update the model weights during the training.
Besides, the settings of betas and weight decay in Adam are uncommon.
# arg
parser.add_argument('--weight_decay', default=5e-3, type=float, help='Weight decay [5e-3]')
# adam
optimizer = torch.optim.Adam(milnet.parameters(), lr=args.lr, betas=(0.5, 0.9), weight_decay=args.weight_decay)

Is there any reason to set the hyperparameters like these (especially the strange settings of betas)?

Hi, you could verify that using lr=0,0001, default betas and 0 weight decay has very minor impacts on the results. Also, these settings are not strange at all and are quite common practices for small batch sizes and small datasets.

@binli123 Got it. I know the learning rate (and other settings) would have a minor impact on the results. However, I am wondering if you claim you use 0.0001 in the paper (but actually use 0.0002 in implementation) would somehow lead to trustworthiness or not.

@binli123 Got it. I know the learning rate (and other settings) would have a minor impact on the results. However, I am wondering if you claim you use 0.0001 in the paper (but actually use 0.0002 in implementation) would somehow lead to untrustworthiness or not.

I think you need to set lr=0.0001 if you want to reproduce the results. The default value of 0.0002 leads to faster convergence and that is for the convenience of the user. If you have any problems or doubts you could easily set this variable to 0.0001 to verify the results instead of posting in an offensive tone here.

@binli123 I see. Thank you very much.

No problem at all. I admit that there are some differences between the paper and the actual github repo, but given my bandwidth, I have tried to make this repo as tidy as I can.


carbsReq pushover improvements

This PR cuts down on spurious/duplicate carbsReq pushover noise:

don't send carbsReq alerts if COB is zero due to clock data parsing error
run pushover from pump-loop immediately after completing a loop
check Nightscout for pushover snoozes (created when another rig has already sent a pushover) immediately before sending one

The latter two combine to eliminate the race condition where two rigs would both send pushover alerts if they completed loops within a minute or two of each other (before cron had a chance to run pushover, update the snooze file to Nightscout, and then run ns-loop to download it to the other rig).

@scottleibrand these changes look good, do you need them testing?

I think they are ready for broader testing, yes. We have been using this branch on all our rigs and it’s looking good so far. Have kept the PR open until I stop finding more things to fix. :-)


chore: [CHAOS-4866]: Add global blackhole chaos tunable docs

Harness Developer Pull Request
Thanks for helping us make the Developer Hub better. The PR will be looked at
by the CODEOWNERS.
What Type of PR is This?

[ ] Issue
[ ] Feature
[ ] Maintenance/Chore

If tied to an Issue, list the Issue(s) here:

Issue(s)

House Keeping
Some items to keep track of. Screen shots of changes are optional but would help the maintainers review quicker.

[ ] Tested Locally
[ ] Optional Screenshot.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

Please check the Execution Link of the Pipeline for the Website Draft URL. This is located in the Preview Step behind the Harness VPN and also is available in #hdh_alerts. E.g Website Draft URL: https://unique-id--harness-developer.netlify.app. Current Draft URL is: https://66024b62829f009660c64863--harness-developer.netlify.app

Please check the Execution Link of the Pipeline for the Website Draft URL. This is located in the Preview Step behind the Harness VPN and also is available in #hdh_alerts. E.g Website Draft URL: https://unique-id--harness-developer.netlify.app. Current Draft URL is: https://660275581d391aafab434512--harness-developer.netlify.app


Node.js  Loaders Team Meeting 2021-08-06

Time
UTC Fri 06-Aug-2021 17:00 (05:00 PM):

Timezone
Date/Time

US / Pacific
Fri 06-Aug-2021 10:00 (10:00 AM)

US / Mountain
Fri 06-Aug-2021 11:00 (11:00 AM)

US / Central
Fri 06-Aug-2021 12:00 (12:00 PM)

US / Eastern
Fri 06-Aug-2021 13:00 (01:00 PM)

EU / Western
Fri 06-Aug-2021 18:00 (06:00 PM)

EU / Central
Fri 06-Aug-2021 19:00 (07:00 PM)

EU / Eastern
Fri 06-Aug-2021 20:00 (08:00 PM)

Moscow
Fri 06-Aug-2021 20:00 (08:00 PM)

Chennai
Fri 06-Aug-2021 22:30 (10:30 PM)

Hangzhou
Sat 07-Aug-2021 01:00 (01:00 AM)

Tokyo
Sat 07-Aug-2021 02:00 (02:00 AM)

Sydney
Sat 07-Aug-2021 03:00 (03:00 AM)

Or in your local time:

https://www.timeanddate.com/worldclock/fixedtime.html?msg=Node.js+Foundation+Loaders Team+Meeting+2021-08-06&iso=20210806T1700
or https://www.wolframalpha.com/input/?i=05PM+UTC%2C+Aug+06%2C+2021+in+local+time

Links

Minutes Google Doc: https://docs.google.com/document/d/1aUgS9e5WvKmHNFIFj4TOVTPyO04BDfOEwYcP6rR_1O8/edit

Agenda
Extracted from loaders-agenda labelled issues and pull requests from the nodejs org prior to the meeting.
nodejs/node

esm: consolidate ESM Loader methods #37468

Invited

Loaders team: @nodejs/loaders

Observers/Guests
Notes
The agenda comes from issues labelled with loaders-agenda across all of the repositories in the nodejs org. Please label any additional issues that should be on the agenda before the meeting starts.
Joining the meeting

link for participants: https://zoom.us/j/97506450082

For those who just want to watch: https://www.youtube.com/c/nodejs+foundation/live

I can’t attend this week. Could someone else run the meeting or could we postpone a week to the next Friday?

I can run (but I think someone with higher permissions would need to handle the Zoom stuff).
I could also do next Friday.

@bmeck @Qard can y'all make it?


[DO NOT REVIEW] Log diagnostic events

What

Why

How Tested

Process & policy checklist

[ ] I have updated the project documentation to reflect my changes if necessary.
[ ] I have read the CONTRIBUTING documentation.

Is this a breaking change?

[ ] This change causes current functionality to break.

Only intended to share some debugging setup.


Problems extracting odata

I really don't know if this is the right place to write my problem here. If not, pardon me!
I tried out the library but I ran into the following error:

I'm quite sure I did everything right (and I also doublechecked the providing oDataService if the values were set right). Also username and password are right.
This is my code:
`from odata import ODataService
from requests.auth import HTTPBasicAuth
import requests
url = 'http://source.com/cdsodata/'
my_auth = HTTPBasicAuth('XXX', 'YYY')
Service = ODataService(url, auth=my_auth)
default = 'Geids'
Order = Service.entities.get('Geids', default)
query = Service.query(Order)
query = query.limit(100)
for entity in query:
print(entity.Geid)`
Maybe you can help me?

Your issue lies in the following two lines, Service.entities is a list of odata.entity.Entity and you specified your default fallback value to be a str object.
default = 'Geids'
Order = Service.entities.get('Geids', default)

This is due to the fact the odata library wasn't able to read your XML response correctly (this was a recurring issue for me, I eventually ended up changing the XML namespaces under odata.metadata.MetaData.namespaces for it to work) or your server doesn't have any Geids entities, hope this helps

if 'Geids' not in Service.entities:
    print('No geids entity definition found.')
    exit()

Order = Service.entities['Geids']

Hi,
Thank you for your help. The server definitely has some entities here.
Where exactly did you change the XML namespace? In the "metadata.py" ?

Hi @tpow ,
sorry for the late answer, but I was on vacation.
I've added the reflect_entities but still no effect.
The location of the metadata is the right one:

So that can't be the issue here.
Ok, but all entities are in json format, so that can't be neither the problem. Here are two examples of the entities (the first from https://source/cdsodata and the second from the Geids):

Your examples all have $metadata without a trailing slash. Does your OData site return the metadata if you add a slash after $metadata like /cdsodata/$metadata/? If not, you'll need pull request #26 by nicenemo.
Tim

Hi @4quen,
Perhaps enabling logging would give you some insight? Here's code similar to what I'm using to test things:
import logging
import odata
import requests
username = "XXX"
password = "YYY"
url = "https://source.com/cdsodata/"
with requests.Session() as s:
     s.auth = (username, password)
     logging.basicConfig(level=logging.DEBUG)
     log = logging.getLogger("odata")
     Service = odata.ODataService(url, session=s, reflect_entities=True)
     print(Service)
     print(len(Service.entities))

     for ent in Service.entities:
         print(ent)

If you're using basic auth, the above should work for you.
Also, in a web browser does both
https://source.com/cdsodata/$metadata/
and
https://source.com/cdsodata/$metadata
give you the same results? (It should be providing XML)
Also, does the following give JSON results?
https://source.com/cdsodata/Geids
Based on your examples, it should.
Tim

Hi, I activated the log and received the following lines:

INFO: odata.metadata:Loading metadata document: http://source.com/cdsodata/$metadata/
DEBUG: urllib3.connectionpool:Starting new HTTP connection (1): source.com:80
DEBUG: urllib3.connectionpool:http://source.com:80 "GET /cdsodata/$metadata/ HTTP/1.1" 403 1233
INFO: odata.metadata: Loaded 0 entity sets, total 0 types
<ODataService at http://source.com/cdsodata/>
0

And yes all of your three examples showed the expected results.

The 403 error indicates that the problem isn't with python-odata, but with your credentials, the access method, or the url. Perhaps you can't connect to that site using basic auth? If you need to authenticate before loading the $metadata url with a browser then you'll need to mimic that before python-odata will work. You might try connecting using the Requests module first and once you get that working I suspect python-odata will work fine.
In my previous example, I'd try replacing the Service = odata.ODataService line and everything following it with something like this:
     response = s.get(url + '$metadata/')
     print(response)
     response.raise_for_status()
     print(response.text)

You can then make changes to your authentication process until the response prints <Response [200]> and you see the xml from the metadata. For example, you should double check the credentials and url. If you're filling out a form in your browser before you can access the odata provider, you might need to post to the form URL (s.post) and then call the ODataService.
Good luck!
Tim


Decouple things from common that aren't common

And put them in their relevant apps.

Dupe of https://github.com/wellcomecollection/wellcomecollection.org/issues/6927?

Closing as dupe.


Testing sent founds visibility in multi-test

closes #347

I agree with merging those refactor in separate PR, but I prefer this one to be the feature one (better name, better tracking), so I just created separate one with those changes only: #365

Please ping me on discord when ready for the final review


How to modify random text noise for custom image noise?

I am going to train the image to remove the watermark.
I have some transparent watermark images now.
what should I do?
How should I change the random text overlay to a custom watermark overlay?

This repo is not suitable for this kind of purposes.
The main purpose of this repo is to reproduce the idea of noise2noise training, not to create useful denoising models.
There should be more suitable denoising implementations other than this repo.


Why does MemoryLocation not contain CpuOnly?

The VMA has a option for HOST_VISIBLE and HOST_COHERENT memory types by using CPU only. Otherwise I have to use CPU to GPU which is not what I require.

CpuOnly would be a bit of a misnomer - since all resources allocated through our allocator (and through VMA) are also visible to the device, so we've omitted it on purpose.
The memory that matches those flags that is closest to what you're looking for is GpuToCpu, which asks for HOST_VISBLE | HOST_COHERENT | HOST_CACHED, meaning it's optimized for readback on the CPU.
What is the use-case that you have specifically for CpuOnly?

Ahh, thank you! That makes sense, I though it was the only setting to have HOST_VISIBLE and HOS_COHERENT to be filled out. I was porting code from VMA over to this crate because VMA rust seems not have active contributions. I was allocating index buffers and vertex buffers. Unrelated to memory settings but would a depth buffer be consider linear for the AllocationCreateDesc?

As a rule of thumb nothing should be marked linear, especially not render targets or depth/stencil targets.
Linear is a niche feature for reading back textures on the CPU.


grouping layers

hey jalagar, I have a question about generating gifs with different layers and body.
for my example, lets talk about cars.
so i want to have 3 different model of cars (Tesla, Porcshe, Lambo) and each of them will have their own set of wheels.
Tesla has 3 wheel designs
Porcshe has 2 wheel designs
Lambo has 4 wheel designs
but all of the cars will fall under the same background.
how can I solve that using your tool?
I have checked the layer grouping example but it wasn't that clear for me to proceed.
waiting eagerly for your reply! Thanks :)

Would it be an easy fix to just put that background with the Tesla group, Porcshe group, and Lambo group?

I would use if-then statements not grouping. Grouping is good to group entire layers together, in your case you want a
if tesla then 3 wheel, if porcshe then 2 wheel etc. Nice to hear from you again!

Check out the readme and the layers_if_then for an example

I'm going to close this issue @Neonode22, feel free to re-open if you have any other questions.


custom paths in settings schema

Summary

Adding option buildtest show schema --settings to show settings schema
Add config section in settings schema with subsection paths where we can define the test directory, location where to search buildspecs which is a colon separate list that mimics $PATH but for buildspecs, and location where to clone repos via buildtest get.

Here is my configuration file
(buildtest) siddiq90@DOE-7086392 buildtest % buildtest config view
editor: vi
executors:
  runlocal:
    type: local
    description: submit jobs locally
config:
  paths:
    get_repo_path: /tmp/buildtest/repos
    testdir: /tmp/buildtest/tests
    buildspec_path: /tmp/buildtest/repos

Custom Clone Path get_repo_path
(buildtest) siddiq90@DOE-7086392 buildtest % buildtest get https://github.com/buildtesters/buildtest-cori
Cloning into '/private/tmp/buildtest/repos/github.com/buildtesters/buildtest-cori'...
remote: Enumerating objects: 9, done.
remote: Counting objects: 100% (9/9), done.
remote: Compressing objects: 100% (9/9), done.
remote: Total 9 (delta 0), reused 9 (delta 0), pack-reused 0
Unpacking objects: 100% (9/9), done.

If get_repo_path is not set it will default to original path as implemented before
(buildtest) siddiq90@DOE-7086392 buildtest % buildtest get https://github.com/buildtesters/buildtest-cori
Cloning into '/Users/siddiq90/.buildtest/site/github.com/buildtesters/buildtest-cori'...
remote: Enumerating objects: 9, done.
remote: Counting objects: 100% (9/9), done.
remote: Compressing objects: 100% (9/9), done.
remote: Total 9 (delta 0), reused 9 (delta 0), pack-reused 0
Unpacking objects: 100% (9/9), done.

Customize testdir
If testdir defined in config it will take precedence, here is a sample testdir when running buildtest build -b tests/examples/buildspecs/environment.yml
buildtest) siddiq90@DOE-7086392 buildtest % ls -l /tmp/buildtest/tests
total 8
-rwxr-xr-x  1 siddiq90  wheel   87 May 27 08:45 hello_dinosaur.sh
drwxr-xr-x  6 siddiq90  wheel  192 May 27 08:45 run

If --testdir passed on command line, this overrides configuration.
buildtest) siddiq90@DOE-7086392 buildtest % buildtest build -b tests/examples/buildspecs/environment.yml --testdir /tmp/random

            Discovered Buildspecs

/Users/siddiq90/Documents/buildtest/tests/examples/buildspecs/environment.yml

Buildspec Name                 SubTest                        Status                         Buildspec Path
________________________________________________________________________________________________________________________
environment                    hello_dinosaur                 PASS                           /Users/siddiq90/Documents/buildtest/tests/examples/buildspecs/environment.yml

============================================================
                        Test summary
============================================================
Executed 1 tests
Passed Tests: 1/1 Percentage: 100.000%
Failed Tests: 0/1 Percentage: 0.000%
(buildtest) siddiq90@DOE-7086392 buildtest % ls -l /tmp/random
total 8
-rwxr-xr-x  1 siddiq90  wheel   87 May 27 08:48 hello_dinosaur.sh
drwxr-xr-x  4 siddiq90  wheel  128 May 27 08:48 run

Buildspec Paths
This is probably the most useful feature and i think having user be able to customize search path will help autocomplete full path by passing relative path. So with original configuration as mentioned above we can reference a buildspec now using the search path
(buildtest) siddiq90@DOE-7086392 buildtest % buildtest build -b github.com/buildtesters/buildtest-cori/system/mountpoint.yml

            Discovered Buildspecs

/private/tmp/buildtest/repos/github.com/buildtesters/buildtest-cori/system/mountpoint.yml

Buildspec Name                 SubTest                        Status                         Buildspec Path
________________________________________________________________________________________________________________________
mountpoint                     mount_check                    FAIL                           /private/tmp/buildtest/repos/github.com/buildtesters/buildtest-cori/system/mountpoint.yml

============================================================
                        Test summary
============================================================
Executed 1 tests
Passed Tests: 0/1 Percentage: 0.000%
Failed Tests: 1/1 Percentage: 100.000%

We can add extra search paths so that user don't have to specify the full path relative from github.com ... such as one below
(buildtest) siddiq90@DOE-7086392 buildtest % cat ~/.buildtest/settings.yml| grep buildspec
    buildspec_path: /tmp/buildtest/repos/github.com/buildtesters/buildtest-cori:/private/tmp/buildtest/repos/github.com/buildtesters/buildtest-stampede2

(buildtest) siddiq90@DOE-7086392 buildtest % buildtest build -b system/mountpoint.yml

            Discovered Buildspecs

/private/tmp/buildtest/repos/github.com/buildtesters/buildtest-cori/system/mountpoint.yml

Buildspec Name                 SubTest                        Status                         Buildspec Path
________________________________________________________________________________________________________________________
mountpoint                     mount_check                    FAIL                           /private/tmp/buildtest/repos/github.com/buildtesters/buildtest-cori/system/mountpoint.yml

============================================================
                        Test summary
============================================================
Executed 1 tests
Passed Tests: 0/1 Percentage: 0.000%
Failed Tests: 1/1 Percentage: 100.000%

Basically we get the first entry in search path that is a hit and process the buildspec which is how $PATH behaves. If there are two files with same name then whichever search tree was defined first will be honored. Since buildspec_path is not required if its not defined it will honor the default path $HOME/.buildtest/site

@shahzebsiddiqui I won't have time in the next month or two to continue reviewing, it's very arduous. Please find someone else to review in the meantime. Thanks!

Thank you for your time and contribution to buildtest, your feedback has been very helpful and insightful over the past 3 months. Sure, i will figure something out in the interim, i can totally understand your situation i don't have the bandwidth to contribute to projects like you do. I wish you the best, let's stay in touch :-)


Feature/issue 205 - Add Confluence API key

GitHub Issue: #205
Description
Define a usage plan and API key for the Confluence workflow.
Use a single usage plan starting with the following limits:

Limit: 11,107,815 requests per month + additional 1,000,000 for testing
Burst limit: 3,000
Rate limit: 6,000

Implement one API key per trusted partner to help track and isolate usage.
Overview of work done

Modify Terraform API gateway API key and usage plan.
Modify Terraform to store a list of trusted API keys in SSM parameter.
Modify Lambda Authorizer code to read in the API key list from the SSM parameter and determine if the x-hydrocron-key header is in the list. This will return the trusted partner policy and count against the usage plan limits for the provided API key.
Updated API key documentation to point users to open a GitHub issue to obtain a key.
Updated internal wiki documentation for adding an API key.

Overview of verification done

Updated authorizer tests to test for trusted key list.

New and existing unit tests pass.
Overview of integration done
Deployed to SIT environment and confirmed creation of API key and update to usage plan limits.
Ran several tests to determine how API keys work when two are assigned to the same usage plans. Confirmed that you can define two API keys and assign them to one usage plan and API requests associated with one key does not impact the limits of the other key.
Ran Lambda authorizer test in the SIT environment and confirmed expected execution for default and trusted keys. Also tested adding an additional key to make sure added API key is detected by Lambda authorizer.
PR checklist:

[X] Linted
[X] Updated unit tests
[X] Updated changelog
[X] Integration testing

See Pull Request Review Checklist for pointers on reviewing this pull request

I have fully tested adding API keys and am ready to finalize this PR.
However, I wanted to discuss a cycle error I keep running into when deploying the feature/issue-205 branch to SIT and then deploying the develop branch to SIT. Here is the error:
│ Error: Cycle: aws_ssm_parameter.trusted-user-parameter, aws_api_gateway_usage_plan.trusted-user-usage-plan (expand), aws_api_gateway_usage_plan_key.trusted-user-usage-key (expand), aws_api_gateway_usage_plan_key.trusted-user-usage-key, aws_ssm_parameter.default-user-parameter (expand), aws_ssm_parameter.default-user-parameter, aws_api_gateway_api_key.default-user-key, aws_ssm_parameter.hydrocron-api-url (expand), aws_ssm_parameter.hydrocron-api-url, aws_api_gateway_usage_plan.default-user-usage-plan (expand), aws_api_gateway_usage_plan.default-user-usage-plan, aws_api_gateway_deployment.hydrocron-api-gateway-deployment (expand), aws_api_gateway_deployment.hydrocron-api-gateway-deployment, data.aws_iam_policy_document.lambda-invoke-authorizer-policy, data.aws_iam_policy_document.lambda-invoke-authorizer-policy (expand), aws_iam_role.hydrocron-gateway-authorizer-role (expand), aws_lambda_function.hydrocron_lambda_authorizer, aws_api_gateway_rest_api.hydrocron-api-gateway (expand), aws_api_gateway_stage.hydrocron-api-gateway-stage, aws_api_gateway_deployment.hydrocron-api-gateway-deployment (destroy deposed 82d9390d), null_resource.api_key_hash (destroy), aws_iam_role.hydrocron-gateway-authorizer-role, aws_api_gateway_rest_api.hydrocron-api-gateway, aws_api_gateway_stage.hydrocron-api-gateway-stage (expand), aws_api_gateway_usage_plan.trusted-user-usage-plan

Seen in these logs: https://github.com/podaac/hydrocron/actions/runs/10379628958/job/28738315949
The only way I have found to resolve this is to redeploy feature/issue-205 several times and then develop can be deployed successfully. I can't quite track down where this is occurring. It does seem to happen when I test out adding a new API key and then when the develop Terraform executes and tries to destroy it.
Theoretically, we shouldn't have this issue moving forward from the feature/issue-205 branch but am wondering if this would be an issue if we decided to remove an API key?
@frankinspace and @torimcd - what do you think? Any ideas on how to isolate the cause?


should fastdigest match python's spooky?

python's spooky, which uses the Hash128 function of the original SpookyHash source, shows a different value to fastdigest when I hexlify the output for a given string input. Should these match given you're using the streaming version of SpookyHash? Is there another reason why they don't match? I am trying to incorporate this into digest and I'm guessing the author will insist that the default output matches something canonical, if such a thing exists, for a few strings.
I have changed the fastdigest seeds to 0 (which is what the python package uses by default) and rebuilt.
You convert the two final 64-bit integer output to a string using:
    SEXP ans;
    PROTECT(ans = allocVector(RAWSXP, 16));
    unsigned char *tmp;
    tmp  = (unsigned char *) &h1;
    for(int i = 0; i < 8; i++)
	RAW(ans)[i] = tmp[i];
    tmp = (unsigned char *) &h2;
    for(int j = 0; j < 8; j++)
	RAW(ans)[j + 8] = tmp[j];
    UNPROTECT(1);
    return ans;

and the python package converts the output of SpookyHash's Hash128 function to a 128-bit integer using:
PyObject *retval = _PyLong_FromByteArray((unsigned char *)seed, 16, 1, 0);
    return retval;

and I convert that value to a string using:
hexlify(hash128("abc").to_bytes(16, 'little'))

matches in the digest version now:
https://github.com/eddelbuettel/digest/pull/100


Support stack transformations

Hello!

Vote on this issue by adding a 👍 reaction
If you want to implement this feature, comment to let us know (we'll work with you on design, scheduling, etc.)

Issue details
This is a feature supported in other language SDKs but missing from the Java SDK.
https://www.pulumi.com/docs/intro/concepts/resources/options/transformations/#stack-transformations

Affected area/feature

Unfortunately, setting transformations on the context is not possible with the current implementation of Resource.
Since Stack inherits Resource initialization logic, the transformations are registered in the constructor.
Also the Stack needs to exist before any user code is executed (including setting the transformations on the context).
Alternative APIs:
dotnet inspired (very easy to implement)
Pulumi.run(ctx -> { ... }, StackOptions.builder().transformations(...).build())

or a "fluent" approach (new public interface will be required for the withStackTransformations to have something to return
Pulumi
    .withStackTransformations(...)
    .run(ctx -> {...}

I personally don't have strong preference.


Deprecate @bigtest/interaction package

Originally, the interactors were published under the @bigtest/interaction package.
We accidentally installed it and were . We should probably publish a version with a deprecation message like
"DID YOU MEAN @bigtest/interactor`?

✅

https://www.npmjs.com/package/@bigtest/interaction

🤔 can we spit something to the console in a post install hook?

We would have to release a new package to include a post install script. I think almost 0 people are using it. Plus NPM does throw a warning:


Update the build.gradle file

Cordova Jira Ticket Exapling the Bug this fixes
https://issues.apache.org/jira/browse/CB-14163
We had trouble using this plugin on android with other plugins that also try to use ext.postBuildExtras.
We had cordova-support-google-services and cordova-plugin-code-push installed as well as this plugin and all three were trying to use postBuildExtras.
But only the last plugin listed in our config file would have its code executed at built time. the link above describes the cause of the issue pretty well

Is there news here? I heard people doing work arounds like this: https://github.com/cloudscapelabs/cordova-support-google-services/blob/csl/build.gradle, but it would be great if this is solved at this level.


Add JSON lint and fix-lint

Resolve #133
Add commands to validate the JSON format in make lint and also fix it in make fix-lint

Great!! :)


Komodo logo too big (KMD)

The new kmd.svg logo is far larger than the other logos. It's got a width and height of 96 instead of 32, and it breaks any consistency when including the logos anywhere.

I've done a workaround in https://github.com/atomiclabs/cryptocurrency-icons/releases/tag/v0.16.1.

@NadiaSotnikova The next time you do new icons, can you also generate a new version of the KMD icon with the correct size (viewBox)? No rush though as it's worked around for now.


add support for conversion to .vtu (Paraview/VTK)

Hi,
I added support for the conversion of the .h5 files from bichon to .vtu files suitable for visualization in Paraview. It's not particularly elegant as a comparison of point locations computes the node ordering, but it gets the job done!

In paraview, one has to use the option "nonlinear subdivision level" to actually see (an approximation of) the higher order mesh:


[stable/prometheus] Add custom pod labels for prom server

What this PR does / why we need it:
This PR makes it possible to add custom, user defined, labels to the prometheus server component.
Which issue this PR fixes
(optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close that issue when PR gets merged)

fixes #

Special notes for your reviewer:
Checklist
[Place an '[x]' (no spaces) in all applicable fields. Please remove unrelated fields.]

[x] DCO signed
[x] Chart Version bumped
[x] Variables are documented in the README.md
[x] Title of the PR starts with chart name (e.g. [stable/chart])

/assign @gianrubio

@mgaggero @gianrubio 👋 any update regarding this?

@mgaggero @gianrubio 👋 any update regarding this?

I guess there is a typo in one of the names.

@mgaggero @gianrubio 👋 any update regarding this?

I guess there is a typo in one of the names.

ah there is. my bad. I meant to tag @mgoodness

/assign
/ok-to-test
/lgtm

/test pull-charts-e2e

/lgtm

/lgtm

/retest

@maorfr Could you please explain to me why this is failing? The details in the provided link don't tell me anything. What I can tell is that the helm deployment timed out? If so could you see why?

/test pull-charts-e2e

/lgtm
nicely done, thanks!


Add Cotx X3 hotspot support

As the necessary parameters has been already added to hm-pyhelper we now need to add it to the base repository and make necessary arrangements.
Acceptance Criteria

[x] Add variant into variant_definitions in hm-pyhelper repository
[x] Create custom inputs (e.g. additional buttons) and outputs (e.g. additional LED's) in the new variant and update the logic in other repositories if necessary.
[x] Release and publish it via CI/CD pipeline
[x] Bump hm-pyhelper for hm-pktfwd, hm-diag and hm-config repositories and push them to create necessary docker images (could take a while) (Example pull request for hm-config in such case)
[x] Create related production Balena fleets (will be called just fleets from now on) as "helium-<device_type>-[variant]-[frequency]-[user]" (parameters in brackets are optional)
[x] Edit configuration parameters of the fleet
[x] Add VARIANT and FREQ environment variables into fleet variables
[x] Update automation steps in this repository for newly created fleets
[x] Add sentry environment variables into fleet variables
[x] Modify generate-images.sh from NebraLtd/hotspot-production-images repo root.
[x] Modify generate-urls.sh from NebraLtd/hotspot-production-images repo root.
[x] Modify generate-images.yml from NebraLtd/hotspot-production-images repo under .github/workflows folder.
[x] Modify settings.ini file if necessary

@vpetersson @marvinmarnold Could you create a test fleet for Cotx?

@vpetersson @marvinmarnold Could you create a test fleet for Cotx? And also helium-cotx

Done

Testnet is looking good. Merging to production.


Participation Request

Hi! Thank you for your PR. Please edit to place your information at the bottom of the file.

oh ok thank you


polyval with timedelta64 coordinates produces wrong results

What happened?
I'm not sure if this is a bug or an expected breaking change, but I'm not able to reproduce the results generated by polyval using a timedelta64 coordinate. The results are correct in xarray=2022.3.0, whereas they are wrong in the latest unreleased version (main, commit 6bb2b855498b5c68d7cca8cceb710365d58e604).
What did you expect to happen?
Both the stable and latest polyval functions should return the same results.
Minimal Complete Verifiable Example
import xarray as xr
import numpy as np

values = np.array(
    [
        "2021-04-01T05:25:19.000000000",
        "2021-04-01T05:25:29.000000000",
        "2021-04-01T05:25:39.000000000",
        "2021-04-01T05:25:49.000000000",
        "2021-04-01T05:25:59.000000000",
        "2021-04-01T05:26:09.000000000",
    ],
    dtype="datetime64[ns]",
)
azimuth_time = xr.DataArray(
    values, name="azimuth_time", coords={"azimuth_time": values - values[0]}
)

polyfit_coefficients = xr.DataArray(
    [
        [2.33333335e-43, 1.62499999e-43, 2.79166678e-43],
        [-1.15316667e-30, 1.49518518e-31, 9.08833333e-31],
        [-2.50272583e-18, -1.23851062e-18, -2.99098229e-18],
        [5.83965193e-06, -1.53321770e-07, -4.84640242e-06],
        [4.44739216e06, 1.45053974e06, 5.29960857e06],
    ],
    dims=("degree", "axis"),
    coords={"axis": [0, 1, 2], "degree": [4, 3, 2, 1, 0]},
)

print(xr.polyval(azimuth_time, polyfit_coefficients))

MVCE confirmation

[X] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray.
[X] Complete example — the example is self-contained, including all data and the text of any traceback.
[X] Verifiable example — the example copy & pastes into an IPython prompt or Binder notebook, returning the result.
[X] New issue — a search of GitHub Issues suggests this is not a duplicate.

Relevant log output
# v2022.3.0 (Correct results)
<xarray.DataArray (azimuth_time: 6, axis: 3)>
array([[4447392.16      , 1450539.74      , 5299608.57      ],
       [4505537.25588366, 1448882.82238152, 5250846.359196  ],
       [4563174.92026797, 1446979.12250014, 5201491.44401733],
       [4620298.31815291, 1444829.59596699, 5151549.377964  ],
       [4676900.67053846, 1442435.23739315, 5101025.78153601],
       [4732975.25442459, 1439797.08038974, 5049926.34223336]])
Coordinates:
  * azimuth_time  (azimuth_time) datetime64[ns] 2021-04-01T05:25:19 ... 2021-...
  * axis          (axis) int64 0 1 2

# v2022.3.1.dev102+g6bb2b855 (Wrong results)
<xarray.DataArray (axis: 3, azimuth_time: 6)>
array([[1.59620685e+30, 1.59620689e+30, 1.59620693e+30, 1.59620697e+30,
        1.59620700e+30, 1.59620704e+30],
       [1.11164807e+30, 1.11164810e+30, 1.11164812e+30, 1.11164815e+30,
        1.11164818e+30, 1.11164821e+30],
       [1.90975722e+30, 1.90975727e+30, 1.90975732e+30, 1.90975736e+30,
        1.90975741e+30, 1.90975746e+30]])
Coordinates:
  * axis          (axis) int64 0 1 2
  * azimuth_time  (azimuth_time) timedelta64[ns] 00:00:00 00:00:10 ... 00:00:50

Anything else we need to know?

Ok, the first idea does not work since values is a numpy array.
The second idea should work, so this is a bug.
It seems that polyval does not work with timedeltas, I will look into that.


Best AutoEncoder

Hello,
For DEC, how can I get the best AutoEncoder after the training is completed? There is one relevant attributes in the current version, saying "Final AutoEncoder". Does "Final" mean the best AutoEncoder?
Thanks.

Hi,
DEC is not trained on multiple autoencoders. Therefore, we do not receive a 'best' autoencoder, but a single autoencoder after it has been updated for a certain amount of epochs.
'Final' just indicates that the DEC training procedure has successfully been applied.
The trained autoencoder can be obtained by the following code:
dec = DEC(n_clusters=10)
dec.fit(X)
ae = dec.autoencoder
If you want to receive multiple autoencoders you have to run DEC multiple times.

Thanks for the clear answer.
I think that my question was not clear, so that I would like to ask you the same question but in a different way. Does the final autoencoder mean that it has optimal parameter values which result in the best representation (i.e., best features) of raw data?
Thanks in advance.

Typically, the best representation depends heavily on your task. Since DEC does not consider the reconstruction loss when optimizing the clustering result - in contrast to, e.g., IDEC -, it will generally output very compact cluster shapes.
Note that DEC is not deterministic.
Therefore, the result of DEC depends very much on the initialization (i.e., on the quality of the pretraining of the autoencoder and the initial clustering result). When running DEC multiple times, you will almost certainly get a different result with a different autoencoder (depending on the seed).
Thus, the autoencoder returned by DEC will most likely not be optimal. DEC just outputs the autoencoder present after X epochs. It can even happen that a better autoencoder with a lower loss was already identified in an earlier epoch that is not returned.
I hope the answer helps you a little.

Thanks for your quick response again.
I have one more question. Then, do other approaches (i.e., DKM and DCN) return an optimal autoencoder after all the training is completed?

No, the behavior of the different deep clustering approaches is highly comparable in that regard.


Source Not Showing

Issue Type: Bug
I cannot view/edit source members anymore on one of my systems?   I can see the source files but when open one, I get a blank page.  This just started in the last month or so.
SQL is not enabled in the settings and the QCCSID is 66535.
Looking at the Code for IBM i output, seems the DB2 command is not returning anything?
/home/NUJKJ: LC_ALL=EN_US.UTF-8 system "call QSYS/QZDFMDB2 PARM('-d' '-i' '-t')"
CREATE OR REPLACE ALIAS ILEDITOR.QGPL_QCLSRC_A_CHGUSR_C for QGPL.QCLSRC("A_CHGUSR_C")
{
"code": 0,
"signal": null,
"stdout": "DB2>",
"stderr": ""
}
/home/NUJKJ: LC_ALL=EN_US.UTF-8 system "call QSYS/QZDFMDB2 PARM('-d' '-i' '-t')"
select srcdat, rtrim(srcdta) as srcdta from ILEDITOR.QGPL_QCLSRC_A_CHGUSR_C
{
"code": 0,
"signal": null,
"stdout": "DB2>",
"stderr": ""
}
Extension version: 1.5.2
VS Code version: Code 1.69.2 (3b889b090b5ad5793f524b5d1d39fda662b96a2a, 2022-07-18T16:12:52.460Z)
OS version: Windows_NT x64 10.0.22000
Restricted Mode: No

System Info

Item
Value

CPUs
11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz (8 x 1382)

GPU Status
2d_canvas: enabledcanvas_oop_rasterization: disabled_offdirect_rendering_display_compositor: disabled_off_okgpu_compositing: enabledmultiple_raster_threads: enabled_onopengl: enabled_onrasterization: enabledraw_draw: disabled_off_okskia_renderer: enabled_onvideo_decode: enabledvideo_encode: enabledvulkan: disabled_offwebgl: enabledwebgl2: enabled

Load (avg)
undefined

Memory (System)
15.74GB (3.86GB free)

Process Argv
--crash-reporter-id 3753743e-64a9-46d2-8a37-ce0176bfb9ca

Screen Reader
no

VM
0%

A/B Experiments
vsliv368cf:30146710
vsreu685:30147344
python383:30185418
vspor879:30202332
vspor708:30202333
vspor363:30204092
vslsvsres303:30308271
pythonvspyl392:30443607
vserr242cf:30382550
pythontb:30283811
vsjup518:30340749
pythonptprofiler:30281270
vshan820:30294714
vstes263cf:30335440
pythondataviewer:30285071
vscod805cf:30301675
binariesv615:30325510
bridge0708:30335490
bridge0723:30353136
vsaa593cf:30376535
vsc1dst:30438360
pythonvs932:30410667
wslgetstarted:30449410
vscscmwlcmt:30465135
cppdebug:30492333
vscaac:30438847
pylanb8912:30529769
vsclangdf:30486550
c4g48928:30535728

@jimjepson Let's try and find the root cause.

First off, launch into a pase shell (you can do this in VS Code using the Terminals button in the status bar when connected to a system)
Run this and share the output: echo "select * from sysibm.sysdummy1" | LC_ALL=EN_US.UTF-8 system "call QSYS/QZDFMDB2 PARM('-d' '-i' '-t')"

When I run it on pub400, this is what I get and you should get something similar.
$ echo "select * from sysibm.sysdummy1" | LC_ALL=EN_US.UTF-8 system "call QSYS/QZDFMDB2 PARM('-d' '-i' '-t')"
DB2>
  ?>

IBMREQD
--------
Y

  1 RECORD(S) SELECTED.

Here is what I get:
echo "select * from sysibm.sysdummy1" | LC_ALL=EN_US.UTF-8 system "call QSYS/QZDFMDB2 PARM('-d' '-i' '-t')"
DB2>
$

@jimjepson i also need you to do this through a pase shell that is not QP2TERM. You can SSH into IBM i through any client (like putty), but also through VS Code like I mentioned before.

Here are the results from putty:
$ echo "select * from sysibm.sysdummy1" | LC_ALL=EN_US.UTF-8 system "call QSYS/QZDFMDB2 PARM('-d' '-i' '-t')"
DB2>
$

@jimjepson this is likely an issue with some SSHD setup. I've seen this before and can share more details on or before Monday.

@jimjepson Check out this comment and the one after it. It's the exact problem you were having.
https://github.com/halcyon-tech/vscode-ibmi/issues/562#issuecomment-1132559069

I stoped and restarted the SSHD server with ENDTCPSVR SERVER(*SSHD) followed by a STRTCPSVR SERVER(*SSHD).   It is working again!
Thank you!


How do you use authMiddleware?

I want to set the auth token after login/register process using the token received
is it possible? do u have an example?

Just provide auth middleware to network layer with token thunk. This function will be called every time, when Relay makes request. You may store your token in global var, localStorage. In my project I use https://www.npmjs.com/package/store
Relay.injectNetworkLayer(new RelayNetworkLayer([
  authMiddleware({
    token: () => window.MyGlobalVarWithToken, // store.get('jwt'),
  }),
]));

thanks, I will use AsyncStorage.
calling it every time is not a problem?

I don't sure that token: async () => await storage('token'), will work at all.
But if does, then you'll get delays before relay send queries to server.
So if possible, try to put token in some variable with sync access.
BTW don't forget about tokenRefreshPromise option in authMiddleware. This promise will be called only if your server returns 401 response. After resolving of token, it implicitly make re-request.
At the end, you may write your own middleware for working with your token logic. As example you may see this simple MW: https://github.com/nodkz/react-relay-network-layer/blob/master/src/middleware/perf.js
You may change req.headers = {} before call next(req).

thanks for all the help, I will try to write a middleware

Few minutes ago I publish new version, where was added allowEmptyToken options to authMiddleware. If allowEmptyToken: true, and token is empty, request proceed without Authorization header.
This can help you, if you do requests to graphql server without token and don't want send empty auth header.

it could be very useful if we could just call a function on authMiddleware to save the token or remove it

Assumption:
Suppose, such function exists. So you should somehow save reference to this function in some global variable. When you log in, you take this var with function and call it. So I can not imagine this somehow realization should be implemented.
Current realization:
Let this variable save not reference to function which you wish, let it keep token. So just provide this variable to arrow function for token option. When you log in, you just write token to this global var. And with next Relay request it reads token from this global var.
I try keep MW realization as simple, as possible. If I begin store token internally, I should also provide functions for reading and removing token ;).

I think it is better to use Relay.Environment to reset relay data on login, thanks for all the support

How I actually use auth middleware:
class ClientApi {
  ...
  getRelayNetworkLayer = () => {
    return new RelayNetworkLayer([
      authMiddleware({
        token: () => this._token,
        tokenRefreshPromise: () => this._getTokenFromServer(this.url('/auth/jwt/new')),
      }),
  ...
}

refresh token promise has the following view
  _getTokenFromServer = (url, data = {}) => {
    console.log('[AUTH] LOAD NEW CABINET JWT');
    const opts = {
      method: 'POST',
      headers: {
        'Accept': 'application/json', // eslint-disable-line
        'Content-Type': 'application/json',
      },
      body: JSON.stringify(data),
    };
    return fetch(url, opts)
      .then(res => {
        if (res.status > 299) { throw new Error(`Wrong response status ${res.status}`); }
        return res;
      })
      .then(res => res.json())
      .then(json => {
        this._token = json.token;
        this._saveTokenToStorage();
        return this._token;
      })
      .catch(err => console.error('[AUTH] ERROR can not load new cabinet jwt', err));
  };

So if new user makes request to server, it responds with 401 header. AuthMW implicitly catchs this header and run  _getTokenFromServer. Internally it makes new request to obtain new token from server, and if succeed, store it to localStorage and pass new token to AuthMW. AuthMW makes re-request and seamlessly pass data to Relay.
So app logic does not know anything about authorization.
Our ClientApi also contains following methods loginByOauth
(providerName), loginByEmail(...), logout(). And instance of ClientApi stored in global variable.
Our ClientApi class was written inspired by https://github.com/reindexio/reindex-js/blob/master/src/index.js

for every 401 it will try to get the token?
how _getTokenFromServer get data parameter?
the problem with this approach is that you do not clean the Relay cache, so if an user logged in, then logout, and another user login, this second user could have access to some left data from the first user

Yep, for every 401 request it tries to refresh token calling _getTokenFromServer, and made re-request. If second request produce again 401 error it will be thrown upper.
FYI you may open dialog window with login and password, send it server, get token, and after that resolve Promise from _getTokenFromServer.
data parameter is undefined for this case. I use method _getTokenFromServer  from another methods like this ClientApi.switchUser(), loginByEmail(), logout(), loginByOauth(). This methods provide some data for auth server for generating proper token.
In my case I just reload page after logout(). For switchUser() after obtaining token, I just force reload some counters after call, in my case no need clear entire relay store.

reload a page is not a good user experience, and it does not work on react native either

Also trying to solve this issue on React Native. Did anyone find a good solution?

Happy to keep this (work in progress) React Native sample public as a template, but how to fix the middleware implementation?
https://github.com/allpwrfulroot/testMiddleware

@sibelius, to address the problem avoiding page refresh, the solution provided by Relay is to create a new Store (see RelayEnvironment) which clean all the cache and avoid one user to see data it should not.
react-router-relay and Relay.Renderer allow you to provide this environment prop.

This could be useful to easily reset a RelayEnvironment
class RelayStore {
  constructor() {
    this._env = new Environment();
    this._networkLayer = null;
    this._taskScheduler = null;

    RelayNetworkDebug.init(this._env);
  }

  reset(networkLayer) {
    // invariant(
    //   !this._env.getStoreData().getChangeEmitter().hasActiveListeners() &&
    //   !this._env.getStoreData().getMutationQueue().hasPendingMutations() &&
    //   !this._env.getStoreData().getPendingQueryTracker().hasPendingQueries(),
    //   'RelayStore.reset(): Cannot reset the store while there are active ' +
    //   'Relay Containers or pending mutations/queries.'
    // );

    if (networkLayer !== undefined) {
      this._networkLayer = networkLayer;
    }

    this._env = new Environment();
    if (this._networkLayer !== null) {
      this._env.injectNetworkLayer(this._networkLayer);
    }
    if (this._taskScheduler !== null) {
      this._env.injectTaskScheduler(this._taskScheduler);
    }

    RelayNetworkDebug.init(this._env);
  }

  // Map existing RelayEnvironment methods
  getStoreData() {
    return this._env.getStoreData();
  }

  injectNetworkLayer(networkLayer) {
    this._networkLayer = networkLayer;
    this._env.injectNetworkLayer(networkLayer);
  }

  injectTaskScheduler(taskScheduler) {
    this._taskScheduler = taskScheduler;
    this._env.injectTaskScheduler(taskScheduler);
  }

  primeCache(...args) {
    return this._env.primeCache(...args);
  }

  forceFetch(...args) {
    return this._env.forceFetch(...args);
  }

  read(...args) {
    return this._env.read(...args);
  }

  readAll(...args) {
    return this._env.readAll(...args);
  }

  readQuery(...args) {
    return this._env.readQuery(...args);
  }

  observe(...args) {
    return this._env.observe(...args);
  }

  getFragmentResolver(...args) {
    return this._env.getFragmentResolver(...args);
  }

  applyUpdate(...args) {
    return this._env.applyUpdate(...args);
  }

  commitUpdate(...args) {
    return this._env.commitUpdate(...args);
  }

  /**
   * @deprecated
   *
   * Method renamed to commitUpdate
   */
  update(...args) {
    return this._env.update(...args);
  }
}

const relayStore = new RelayStore();

just call:
RelayStore.reset(networkLayer)

This is how I'm doing it:
networkLayer.js
const networkLayer = new RelayNetworkLayer([
  urlMiddleware({
    url: _ => '/graphql',
  }),
  authMiddleware({
    token: () => localStorage.getItem('id_token'),
  }),
], {disableBatchQuery: true});

Root.js
import React from 'react';
import Relay from 'react-relay';
import {
  Router,
  browserHistory,
  applyRouterMiddleware,
} from 'react-router';
import useRelay from 'react-router-relay';
import ReactGA from 'react-ga';
import networkLayer from 'shared/utils/networkLayer';
import routes from './routes';

class Root extends React.Component {
  static childContextTypes = {
    logout: React.PropTypes.func,
  };
  state = {
    environment: null,
  };
  getChildContext () {
    return {
      logout: _ => this.logout(),
    };
  }
  componentWillMount () {
    ReactGA.initialize(process.env.GOOGLE_ANALYTICS_KEY);
    const environment = new Relay.Environment();
    environment.injectNetworkLayer(networkLayer);
    this.setState({environment});
  }
  logout () {
    const environment = new Relay.Environment();
    environment.injectNetworkLayer(networkLayer);
    this.setState({environment});
  }
  logPageView = _ => {
    ReactGA.set({ page: window.location.pathname });
    ReactGA.pageview(window.location.pathname);
  }
  render () {
    return <Router
      render={applyRouterMiddleware(useRelay)}
      history={browserHistory}
      environment={this.state.environment}
      routes={routes}
      onUpdate={this.logPageView}
      key={Math.random()}
    />;
  }
}

LoginPage.js
This page has a method that gets passed to a login form. The login form calls an AuthenticateUserMutation, and uses this function as a callback:
  loginUser = data => {
    localStorage.setItem('id_token', data.jwtToken);
    localStorage.setItem('user_uuid', data.userId);
    this.context.setLoggedIn(true);
    this.context.setUserId(data.userId);
    this.context.router.push('/profile');
  }

LoginForm.js
  processLogin = response => {
    const { authenticateUser: { authenticateUserResult } } = response;

    if (authenticateUserResult && authenticateUserResult.userId) {
      this.props.loginUser(authenticateUserResult);
    } else {
      this.setState({ loginError: 'Email and/or Password is incorrect' });
    }
  }
  loginUser = ({ email, password }) => {
    Relay.Store.commitUpdate(
      new AuthenticateUserMutation({ email, password }),
      { onSuccess: this.processLogin },
    );
  }

core/relay.ts
import { getAuth } from "firebase/auth";
import { Environment, Network, RecordSource, Store } from "relay-runtime";

/**
 * Initializes a new instance of Relay environment.
 * @see https://relay.dev/docs/
 */
export function createRelay(): Environment {
  // Configure a network layer that fetches data from the GraphQL API
  // https://relay.dev/docs/guides/network-layer/
  const network = Network.create(async function fetchFn(operation, variables) {
    const auth = getAuth();
    const headers = new Headers({ ["Content-Type"]: "application/json" });

    // When the user is authenticated append the ID token to the request
    if (auth.currentUser) {
      const idToken = await auth.currentUser.getIdToken();
      headers.set("Authorization", `Bearer ${idToken}`);
    }

    const res = await fetch("/api", {
      method: "POST",
      headers,
      credentials: "include",
      body: JSON.stringify({ query: operation.text, variables }),
    });

    if (!res.ok) {
      throw new HttpError(res.status, res.statusText);
    }

    return await res.json();
  });

  // Initialize Relay records store
  const recordSource = new RecordSource();
  const store = new Store(recordSource);

  return new Environment({ store, network, handlerProvider: null });
}

Then, somewhere at the root level:
import { useMemo } as React from "react";
import { RelayEnvironmentProvider } from "react-relay";
import { createRelay } from "../core/relay.js";
import { AppRoutes } from "../routes/index.js";

/**
 * The top-level (root) React component.
 */
export function App(): JSX.Element {
  const relay = useMemo(() => createRelay(), []);

  return (
    <ThemeProvider theme={theme}>
      <RelayEnvironmentProvider environment={relay}>
        <AppRoutes />
    </RelayEnvironmentProvider>
  );
}

https://github.com/kriasoft/relay-starter-kit


[FEATURE] Expose initial_point_generator and base_estimator settings from skopt.Optimizer in HPS

Is your feature request related to a problem? Please describe.
The current CBO class only allows a subset of the parameters in the skopt.Optimizer class as inputs. For instance, initial_point_generator cannot be passed to CBO although it is a parameter in skopt.Optimizer and is in essence hard-coded to the default value when setting the optimizer in CBO. Similarly, the surrogate_model parameter in CBO sets predefined regressors based on an input string, although skopt.Optimizer allows any sklearn regressor as a base_estimator.
Describe the solution you'd like

Ainitial_point_generator parameter in the CBO class itself that gets passed to the optimizer.
Expansion of surrogate_model to allow the user to pass their own sklearn regressor to the underlying base_estimator parameter of the optimizer.

Additional context
I can make a pull request with these changes since I've implemented these on my own fork, but it would be good to get another pair of eyes across the changes. I've only used CBO in my work so far, but after a quick scan of DBO, it looks like these are also unavailable in that class so it may be a good idea to make the same changes there.

Hello @albertkklam,
I think this is a good idea. To avoid creating new arguments, maybeinitial_points for initial_point_generator could be used, and if a str or object is passed it can be detected and we can make the difference with having None or a list of points.
For the surrogate_model make sure to document how the n_jobs parameter will behave if the surrogate model instance is directly passed. Also, I think we need to be careful with how the data-preprocessing is handled depending on the surrogate model (for example normalisation of collected (x, y) values).
You can create a PR, and I can help review/test it.

Addressed in https://github.com/deephyper/deephyper/pull/157 and https://github.com/deephyper/deephyper/pull/158.


a little refactor query/compiler.js

todo: benchmark this changes

todo2: review todo in code

Too many tests depends on alphabetical sort of columns
also user code possibly do same thing

Too many tests depends on alphabetical sort of columns also user code possibly do same thing

I thought we had already removed alphabetical sorting from insert couple of years ago (or at least from updates) because in some cases actual order of columns in insert / update does matter (IIRC there was some way to write some dependencies between columns which were evaluated in order). Maybe that change was postgresql dialect specific 🤔


BLD: added "SDGevidence" field

Added SDGevidence field to document relevance to the SDGs for each product as a string field that can contain both text and links (I admit this is messy, and should probably be split into different fields). Currently this field is optional because the effort to retroactively add this field to all existing products would be monumental, but we are requiring it for new products to be added. Eventually, we could make it required.

@conradsp the discussed changes have been implemented in 984b5e4


Feature - Manager/HR - Add comment components to "Review" page

Description
Is your feature request related to a problem? Please describe.
Managers and HR need the ability to leave comments on various sections of the Job Poster. We want to add this functionality all on the Review page (last step of JPB).
Describe the solution you'd like
Add a commenting component below each section on the Job Poster Builder Review page. See image (Nic to Add). Only the comments for that section should be shown inside each one.
Describe alternatives you've considered
Adding comment section on each page of the JPB was decided against for now. Those pages are "busy" already and we could need the sections on the Review page anyway because HR doesn't have access to the rest.
Images
Place image attachments (wireframes, etc.) here.

[ ] This task is complex and needs acceptance criteria.

The purpose of this feature is to allow HR Advisors to comment on the Review Poster page without significant changes to the interface.
Once comments are made, users will have the ability to show/hide comments making it more the display more manageable.


llama3.1_8b test bug

Describe the bug
runningpytest --capture=tee-sys tests/test_llama_model.py:
the process is killed when the last output (linear) layer of the model is attempted to be loaded onto the device. The relevant line that crashes is at the end of the init of class TtTransformer in demos/wormhole/llama31_8b/tt/llama_model.py, copied below. I can print the tensor size and such on host. Any ideas on how to debug?
    self.output_weight = ttnn.as_tensor(
        state_dict["output.weight"].permute(1, 0),
        device=device,
        layout=ttnn.TILE_LAYOUT,
        dtype=dtype,
        memory_config=ttnn.DRAM_MEMORY_CONFIG,
        cache_file_name=weight_cache_path / "output.weight",
    )

To Reproduce
Steps to reproduce the behavior:

Go to Commit hash 516a8917fac17649abbc9052b680894b432b4de6 from main branch.
run pytest --capture=tee-sys tests/test_llama_model.py
process gets killed!

Expected behavior
test to pass
Please complete the following environment information:

OS: [e.g. Ubuntu 20.04]
Version of software (main branch : 516a8917fac17649abbc9052b680894b432b4de6 )
N300

additional info from user:
I believe the issue may be because the below file path does not exist -- I was not clear what step generates this file. I did try to pass in None as cache_file_name , that did not seem to make a difference.
"Unable to load the tensor from /tt-metal/models/demos/wormhole/llama31_8b/weight_cache_dir/Meta-Llama-3.1-8B-Instruct/tensor_cache_bfp8/output.weight_dtype_BFLOAT8_B_layout_TILE.bin.  The file does not exist."

This message is fine, the cache will be generated on-demand. The process is probably killed because the host CPU is running out of memory. test_llama_model.py loads the full FP32 weights as well as the BFP8 weights and performs inference on the reference model to perform PCC tests. We haven't tested the minimum host DRAM required but I would guess at least 64GB.
To run the test without the reference model and reduce host memory requirements, set run_ref_pt = False near line 37.
I recommend running the demo scripts to generate model output, test_llama_model.py is really for internal development and CI purposes.

@yieldthought thanks for the detailed response. I will inform the customer.


Fixed issue in Python 3 server

In Python 2.x calling keys() makes a copy of the keys of the dictionary.
In Python 3.x returns an iterator instead, which leads to Runtime Errors
when the dictionary is changed while iterating over its keys.

Perfect! Thanks


Client doesn't print understandable error message when creating components which are being deleted

Deploy example
Delete namespaces
Apply example again:

Gen 5 -> 6    	[+] cluster-us-east#platform#analytics_pipeline#platform
              		 -> hdfs, kafka, spark
              	[+] cluster-us-east#platform#hdfs#platform
              		 -> hdfs
              	[+] cluster-us-east#platform#kafka#platform
              		 -> kafka, zookeeper
              	[+] cluster-us-east#platform#spark#platform
              		 -> spark, zookeeper
              	[+] cluster-us-east#platform#zookeeper#platform
              		 -> zookeeper
              	[+] cluster-us-east#social#twitter_stats#realtime
              		 -> analytics_pipeline, tweepub, tweetics, tweeviz
Waiting for changes to be applied....
[Applying changes]
  [59/59] [====================================================================] 100%   Time: 19s
Error! Revision 5 failed with an error and has not been fully applied
FATA[0019] error

Client exits with empty error message
Server log has:

ERRO[0076] (enforce-5-apply) error while applying action 'action-component-create/cluster-us-east#platform#zookeeper#platform#zookeeper': unable to deploy component instance 'cluster-us-east#platform#zookeeper#platform#zookeeper': rpc error: code = Unknown desc = failed to create resource: configmaps "zk-a-8ap5vlrdbpc00" is forbidden: unable to create new content in namespace east because it is being terminated.

The error is clear - namespace is terminated. You should wait till it'll be terminated completely.

Oh, I see, it's not propagated to the client.

related to the fact that apply logs are not propagated to the client. will create another issue for it. client won't need to display logs directly, it will need to display CMD + HTTP LINK to show logs


Store required libs on model object

Store information of required libs per binary on the Model object if there exists an executable path.
Should close #354 and #355

coverage: 41.981% (+0.1%) from 41.843% when pulling c6864cc63dfb0597a255bca78e78bd706d757c2f on jo-basevi:354-solo-model-req-libs into 105fdc0930a43ff0802c9091ebbc6d8c85d6fa40 on payu-org:master.

Ok, I let it run for a bit with MOM6-CICE6 (I wasn't sure how to do a short run of it so I used qdel after it ran for awhile). Though as required_libs are set during setup so I could check that after payu setup, the required_libs field were set as expected.
Everything still works fine with the ACCESS-OM2 configs

Feel free to merge if you are happy it is good to go @jo-basevi

I've also done some checking because Ezhil was reporting odd unrelated behaviour (that is a different issue), but it all seems to work well with single model and multi-model configs.

I just thought, we probably want to check for storage paths in the use path.
That would be added here
https://github.com/payu-org/payu/blob/105fdc0930a43ff0802c9091ebbc6d8c85d6fa40/payu/schedulers/pbs.py#L112
Ideally we might want to look in the paths of the required_libs, but that isn't known until after the run is submitted to PBS, so not available.

Ok, I've added 'module use' paths to the list to search for storage paths and added it to the tests.

Great, thanks! Good to merge I think.


feat: create sequence backend

Excellent PR! Love the rigor you've put in in the UTs 👍✌🏻
I don't have any major concerns, except -
Please validate backward compatibility of the session file. Can you find a way to write a test which uses an older version of the session file and the newer version (which contains sequences now) and ensure that both can be parsed and both work?
We should have a method which loads the session file which you can use to write this type of test.

Agreed, we shuold be able to add an IT here using the Spanner emulator similar to our existing ones.

@Deep1998 and @manitgupta, added unit tests reading session file into conv(new and old) to ensure an older version is also correctly read. Please check. Thanks


Allow for setting deprecateTypingAliases configuration

Pyright has a configuration option for deprecating aliases being imported from the typing module (e.g., typing.Callable -> collections.abc.Callable). However, setting the value for this setting in python.analysis.diagnosticSeverityOverrides doesn't do anything. It would be great if we could use this config in Pylance.

diagnosticSeverityOverrides applies only to diagnostic rules, not to other configuration options. The deprecateTypingAliases configuration option is not a diagnostic rule. It allows only "true" or "false" values.

diagnosticSeverityOverrides applies only to diagnostic rules, not to other configuration options. The deprecateTypingAliases configuration option is not a diagnostic rule.

This makes sense. My confusion stemmed from the fact that deprecateTypingAlias is listed under diagnostic rules in the Pyright docs.

It allows only "true" or "false" values.

For what it's worth, before posting this request, I had tried all different combinations of values for this configuration under python.analysis.diagnosticSeverityOverrides, including "true"/"false"/true/false/"error"/... Nothing works.
If python.analysis.diagnosticSeverityOverrides isn't the right configuration, then what configuration should I use to enable this diagnostic rule?

My confusion stemmed from the fact that deprecateTypingAlias is listed under diagnostic rules in the Pyright docs.

That's a good point. I will make a note of this and try to make it clearer.

what configuration should I use to enable this diagnostic rule?

Pylance doesn't currently expose those configuration settings through the LSP interface. You'd need to set it in a pyrightconfig.json or pyproject.toml config file.

You'd need to set it in a pyrightconfig.json or pyproject.toml config file.

I wasn't aware of this configuration. This addresses my problem. Thank you very much!


Update the glsl validator link for travis.

According to https://github.com/KhronosGroup/glslang, our previous link should have always worked, but it no longer does.  This pins to a specific version.  We can revert it if the original link is fixed.

The original link has been fixed, so closing this PR.


Allow docs to be generated in separate class from target

I think this is the intention of the target_class option—to allow the docs for an API class to be generated in a different class. The existing implementation didn't support the full split, so I did a quick fix to change that. You can take a look at the spec I added to check that my understanding of the desired behavior is correct.

Please update CHANGELOG, fix the build and squash the commits (rubocop -a will do the job). I'll take a closer look at the code.

@dblock Done. Rubocop made a bunch of random code changes, and forced me to refactor for length as well. So, fair warning, this is now merge conflict bonanza.

Looks good, see my comment above and please squash these commits, too.

Ok, pulled the module out and squashed. I didn't squash the Rubocop refactoring, since those commits are totally unrelated to the actual change. That okay with you?

This looks great, merging.


Support background execution

When a process is executed in the background, it stops when reading from the input stream.
Because of the readInputLoop, the mvnd process is stopped immediately.
This involves checking the SIGTSTP with JLine, stopping the thread and resuming it when the SIGCONT is received.

SIGTSTP has the same problem than #193 .


Multiple Exchange And Exchange Transfers

Is it possible to perform synchronous trades on multiple exchanges and perform exchange to exchange wallet balance transfers? It appears the trade engine and strategies are only able to support a single exchange at the moment and are not able to support wallet transfers.

Not at the moment, but it could be done.
Currently, each bot integrates with 1 exchange to keep things simple... and to avoid compromising other 'exchange trading' should it fail - the bot was originally designed to work on 1 exchange and to fail hard and crash if anything unexpected happened.
You'd need to change the config model to support multiple exchanges (like the markets config) and the Trading Engine would need changing so that its control loop executes markets/strats on each exchange - either sequentially in 1 thread, or perhaps a separate thread for each adapter, depending on your requirements.
The wallet transfer capability would be easier to implement - the Trading API would need updating to support this (plus a suitable default op for the existing adapters) for a given exchange id.
As for strats bridging multiple exchanges - I'm assuming arbitrage is on the cards here - there'd need to be significant change to support this: the Strategy API would need updating to notify the strat which exchange the invocation is for... and the Trading API ops would need to accept the exchange id the strategy is wanting to execute on... the Trading Engine would also need a fair bit of work to route these requests back and forth... not rocket science, but a lot smarter than it is now.
This is all great stuff, and it would push the bot a lot further than the noddy sample Bitcoin trading bot I started out with! Sadly, my day-job is limiting the time I can work on BX-bot at the moment - I'm trying to build an Angular admin app for the bot and learn Ethereum smart contract blockchain stuff in my spare time too - but if someone was to fork this project and take it to the next level, I'd be happy to help whenever I get some spare time.
The Angular app and Ethereum stuff I'm working on should be done by Jan/Feb 2017 - I'll be back on BX-bot then.
HTH!

Closing - currently no plans to support multiple concurrent exchanges.


Add matching FutureOr extensions

Closes Issues https://github.com/shinayser/mocktailx/issues/2

Did you want me to bump the package version or will you handle that?

@kmcgill88 Yeah please! Then I will simply upload a new version to pub. Thanks for the pr man =)

Its online, v0.0.4 =)


OAuth2 Tokens

Can we expect to be able to use an Oauth2 token through passport.js or similar?
inputting actual usernames and passwords is not going to work with 2fa

I've started work web a webserver built on top of this library here: https://github.com/coolaj86/node-pokemap
I have to go to bed for now, but I'll try to resume Saturday. There needs to be a way to serialize and deserialize the pokeio object (mentioned in other issues) before it can be completed.

This is definitely possible but whatever client you use to generate a JWT token will need to conform to very specific settings:

https://github.com/carldanley/node-pokemon-go-api/blob/master/src/login.js#L13-L16
https://github.com/carldanley/node-pokemon-go-api/blob/master/src/login.js#L28

just change playerInfo.accessToken


Error in python tutorial

Here is the screen capture of running the python tutorial. fn --version works, and the Ubuntu guest in VirtualBox is able to browse to http://deb.debian.org, but it redirects the url to https.

scott@scott-VirtualBox:~$ cd pythonfn
scott@scott-VirtualBox:~/pythonfn$ fn --verbose run
Building image pythonfn:0.0.1
Sending build context to Docker daemon  7.168kB
Step 1/8 : FROM python:3.6-slim-stretch
---> b31cb11e68a1
Step 2/8 : WORKDIR /function
---> Using cache
---> bcc9d59a78c3
Step 3/8 : RUN apt-get update && apt-get install --no-install-recommends -qy build-essential gcc
---> Running in 346c6a4b01a3
Err:1 http://deb.debian.org/debian stretch InRelease
Temporary failure resolving 'deb.debian.org'
Err:2 http://security.debian.org/debian-security stretch/updates InRelease
Temporary failure resolving 'security.debian.org'
Err:3 http://deb.debian.org/debian stretch-updates InRelease
Temporary failure resolving 'deb.debian.org'
Reading package lists...
 They have been ignored, or old ones used instead.
Reading package lists...
Building dependency tree...
Reading state information...
E: Unable to locate package build-essential
E: Unable to locate package gcc
The command '/bin/sh -c apt-get update && apt-get install --no-install-recommends -qy build-essential gcc' returned a non-zero code: 100

How to debug this? You may try trace route to apt repos from inside docker
container manually.
However, you may try assigning custom docker image in func.yml
(build_image: ....) that does have all necessary packages (gcc, etc.)
installed.
On Thu, Jun 28, 2018 at 21:24 Scott Zimmerman<EMAIL_ADDRESS>wrote:

I meant my guest os is Ubuntu where I'm running fn. I had the same error
yesterday, is there a way to know what is causing the temporary outage? Is
there a way I can point the demo to use the EU repo?
After installing fn, what components is the demo trying to download?
Thanks
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/fnproject/fn/issues/1100#issuecomment-401145576, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AC5L6_r9M71qMSvzKEJjh5xm_sdvnkdhks5uBS1tgaJpZM4U72lQ
.

It seems off that we’re doing an apt-get install during fn build here, I think the base requirements should be bundled into the respective containers we provide

It seems off that we’re doing an apt-get install during fn build here, I think the base requirements should be bundled into the respective containers we provide

this seems to be the case now


2.0 preview - "scale" (only the number) can go beyond "scaleMax" using CLI for scaling

Rancher versions:
rancher/server: v2.0.0-alpha9
rancher/agent: v2.0-alpha4
Infrastructure Stack versions:
healthcheck: v0.3.3
ipsec: v0.12.3
network-services: network-manager:v0.8.6
scheduler: v1.7.6-netes2-1
kubernetes (if applicable):
Docker version: (docker version,docker info preferred)
Server:
Version:      17.03.1-ce
API version:  1.27 (minimum version 1.12)
Go version:   go1.7.5
Git commit:   c6d412e
Built:        Mon Mar 27 17:05:44 2017
OS/Arch:      linux/amd64
Experimental: false
Operating system and kernel: (cat /etc/os-release, uname -r preferred)
NAME="CentOS Linux"
VERSION="7 (Core)"
ID="centos"
Kernel: 3.10.0-514.6.1.el7.x86_64
Type/provider of hosts: (VirtualBox/Bare-metal/AWS/GCE/DO)
DO
Setup details: (single node rancher vs. HA rancher, internal DB vs. external DB)
Single node rancher
external DB  - Server version: 5.5.52-MariaDB MariaDB Server
Environment Template: (Cattle/Kubernetes/Swarm/Mesos)
Rancher 2.0
Steps to Reproduce:
Rancher CLI Version: v1.0.0-alpha4
Edit service and set min max increment values in UI
./rancher --env <env> scale <service>=<num > maxScale>
In my test scenario: ./rancher --env 1a5 scale faas-rancher=40
Results:
"scale": 40,
"scaleIncrement": 2,
"scaleMax": 10,
"scaleMin": 1,
Actual number of containers stays @ scaleMax, so only the scale number goes over. Still confusing in UI / when using the scale number in external applications.
currentScale is also set to "null" and I think this follows the exact count for number of containers in 1.x.x.
Let me know if you need any more info.

These options no longer exist in native kubernetes-land...


Use observers for removal detection in example

Objective
The removal detection example shows an outdated pattern.
Solution
Show how to do this with observers.

Part of #13928.


Refactor v0.4 suggestions for feature examples

This branch includes some suggestions to enrich the examples of data features and add plot_matrix() method to display heatmaps of connectivity matrices.

Codecov Report

Merging #252 (8e52fab) into refactor_v0.4 (55597dd) will decrease coverage by 0.06%.
The diff coverage is 18.18%.

:mega: This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more
@@                Coverage Diff                @@
##           refactor_v0.4     #252      +/-   ##
=================================================
- Coverage          35.89%   35.84%   -0.06%
=================================================
  Files                 59       59
  Lines               4705     4715      +10
=================================================
+ Hits                1689     1690       +1
- Misses              3016     3025       +9

Impacted Files
Coverage Δ

siibra/features/basetypes/regional_connectivity.py
24.13% <18.18%> (-1.84%)
:arrow_down:

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


Fix kedro 0.18.3 recipe dependencies

Checklist

[x] Used a personal fork of the feedstock to propose changes
[x] Bumped the build number (if the version is unchanged)
[x] Reset the build number to 0 (if the version changed)
[x] Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
[x] Ensured the license file is being packaged.

conda-forge-admin, please rerender

Hi! This is the friendly automated conda-forge-linting service.
I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

Hi @rxm7706 - thanks for getting ahead of these changes. Just a note - MerelTheisenQB changed their username to @merelcht, would you be able to add them back to the maintainer list? Thanks!


Timer related go routine schedule problems under high concurrency

What version of Go are you using (go version)?

$ go version
go version go1.14.7 linux/amd64

Does this issue reproduce with the latest release?
Latest is 1.15, so I think the answer should be No
What operating system and processor architecture are you using (go env)?
go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/wli1/.cache/go-build"
GOENV="/home/wli1/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/wli1/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build258194693=/tmp/go-build -gno-record-gcc-switches"

What did you do?

I write a server to handle requests from clients. I hope for each connection the server should process it in a limited time, for example 10ms, which means no matter has the server done the work or not, it should give a respones to the client when the time reachs 10ms. Pseudo-codes are here:
func RecieveRequest(req Request) {
    // tic here
    done := Process(req) // done is a channel to get the response to send, process usually will be done in 6 ms
    select {
    case result := <-done:
        SendResponse(result)
    case <-time.After(time.Duration(10 * time.Millisecond):
        SendResponse(ServerTimeOut) // ServerTimeOut is a predefined response
    }
    // toc here
}

func Serve() {
    // ... codes ...
    for {
     // ... codes ...
        req := accept()
        go RecieveRequest(req)
     // ... codes ...
    }
    // ... codes ...
}

What did you expect to see?
I use tic-toc to messure the actual time interval, the actual time interval should be within 10ms, or a little bit more than 10ms.
What did you see instead?
under low concurrency, most request is processed normally, but there are still very few(under 0.01%) requests' actual process intervals are like 15ms.
under high concurrency, situation worsened. There are more(above 0.1%) actual intervals are larger than 10ms, and the worst case could reach 50~500 ms, which will cause the client timeout.
I think timers should be correct since there is nothing like tight-loop in the codes, during ervery schedule event the processor would check the timer heap to fire those should be fired. Then the problem seems to be the scheduler itself, the scheduler seems not be able to schedule the timer related goroutines to run immediately. But this is just my guess, I'm not sure.
So is this phenomenon normal? Or I did something wrong?
If my guess is correct, should the timer related routines be pushed on the front of the queue?
Hope someone could help, thanks.

func RecieveRequest(req Request) {
    // tic here
    done := Process(req) // done is a channel to get the response to send, process usually will be done in 6 ms
    select {
    case result := <-done:
        SendResponse(result)
    case <-time.After(time.Duration(10 * time.Millisecond):
        SendResponse(ServerTimeOut) // ServerTimeOut is a predefined response
    }
    // toc here
}

Time = ~10ms + time for SendResponse
You should measure it like ...
func RecieveRequest(req Request) {
    // tic here
    done := Process(req) // done is a channel to get the response to send, process usually will be done in 6 ms
    select {
    case result := <-done:
        SendResponse(result)
    case <-time.After(time.Duration(10 * time.Millisecond):
        // toc here
        SendResponse(ServerTimeOut) // ServerTimeOut is a predefined response
    }
}

Yeah, you are right, but this is async call, the effect should be very small. And the client also find this timeout issue.-------- 原始邮件 --------发件人： Chotepud Teo<EMAIL_ADDRESS>2020年8月14日周五 傍晚6:12收件人： golang/go<EMAIL_ADDRESS>Li wen<EMAIL_ADDRESS>Author<EMAIL_ADDRESS>  题： Re: [golang/go] Timer related go routine schedule problems under high concurrency (#40786)
func RecieveRequest(req Request) {
// tic here
done := Process(req) // done is a channel to get the response to send, process usually will be done in 6 ms
select {
case result := <-done:
SendResponse(result)
case <-time.After(time.Duration(10 * time.Millisecond):
SendResponse(ServerTimeOut) // ServerTimeOut is a predefined response
}
// toc here
}
Time = ~10ms + time for SendResponse
You should measure it like ...
func RecieveRequest(req Request) {
// tic here
done := Process(req) // done is a channel to get the response to send, process usually will be done in 6 ms
select {
case result := <-done:
SendResponse(result)
case <-time.After(time.Duration(10 * time.Millisecond):
// toc here
SendResponse(ServerTimeOut) // ServerTimeOut is a predefined response
}
}
—You are receiving this because you authored the thread.Reply to this email directly, view it on GitHub, or unsubscribe.

The only way to be sure it's the timer and not SendResponse that's taking longer than expected is to time it as such.

Thanks, I will do further test.-------- 原始邮件 --------发件人： Chotepud Teo<EMAIL_ADDRESS>2020年8月14日周五 晚上7:08收件人： golang/go<EMAIL_ADDRESS>Li wen<EMAIL_ADDRESS>Author<EMAIL_ADDRESS>  题： Re: [golang/go] Timer related go routine schedule problems under high concurrency (#40786)
The only way to be sure it's the timer and not SendResponse that's taking longer than expected is to time it as such.
—You are receiving this because you authored the thread.Reply to this email directly, view it on GitHub, or unsubscribe.

@AlexRouSg Hello, thanks for your adivice, I have done the test exactly after the timer is fired. I have edited the description. and the result is same.

See #38860 which is fixed in 1.15 but not in 1.14... #39383
cc @ianlancetaylor

Yes, this looks like a dup of #38860.  Closing as a dup, but please comment if you disagree.


OC-1146 - add showOnlyFails option

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Why should the training and testing logic in BackBone3D be different?

Thank you for sharing your work!
In class TeVoxelBackBone8x, you used different fuction for training and testing.
https://github.com/hailanyi/TED/blob/8c455aca982149f5630f48d2bc6db21b26e63dfe/pcdet/models/backbones_3d/spconv_backbone.py#L736
https://github.com/hailanyi/TED/blob/8c455aca982149f5630f48d2bc6db21b26e63dfe/pcdet/models/backbones_3d/spconv_backbone.py#L593
I wonder why do the training and testing logic have to be different.

The  sequential processing for multiple transformed points is time-consuming. So I combined the tensors to a single tensor for parallel processing, that can slightly speed up the inference.


⚠ Bofrost.de has degraded performance

In 7e6ff7b, Bofrost.de (https://www.bofrost.de) experienced degraded performance:

HTTP code: 200
Response time: 1558 ms

Resolved: Bofrost.de performance has improved in 9a1014f after 312 days, 15 hours, 15 minutes.


Audio fixes and enhancements

Fixed #236 (Asset pool loading bug), added parameters to sound_player in config_validator: action, events_when_played, events_when_stopped

That change to config_validator happened somehow in the merge process.  I certainly did not intend for that line to be removed.  I'll fix it.


Popular graphana dashboards show no data for "Pods Memory usage" and "Pods CPU usage"

Discussed in https://github.com/prometheus-operator/kube-prometheus/discussions/1694

Originally posted by surfingjoe March 21, 2022
Running bare-metal K8s with flannel network and MetalLB as Load Balancer.
Cloned https://github.com/prometheus-operator/kube-prometheus.git just a few days ago.  Created "monitoring" namespace.  Performed Kubectl apply for manifests/setup, waited till all pods are up, then applied  manifests/
Instead of port forwarding, I patched each service to be: "Type" load-balancer considering that I'm using MetalLB.
Everything is operational.  Can successfully deploy containers with services exposed externally by the load balancer MetalLB.  All is well in the world. Can open Prometheus port 9090 & Graphana port 3000  externally from a browser and everything seems operational.  Graphana is seeing Prometheus data, and community dashboards are seeing some of the Prometheus data.
Import popular community dashboards like Community dashboard number 315 for example.  Oops, the dashboard reports "No Data" for PODS memory and CPU usage, but Graphana can see Cluster Memory & CPU usage. The dashboard can see Network I/O pressure, but "No data" for Pods network I/O.
As I'm a newbie to Graphana, I'm hoping someone wiser than me can help :)

Same issue here, it seems that some metrics referenced from the dashboard queries like node_namespace_pod_container are not available/exported by the metrics-server

+1 on this, cpu+memory and other metrics are not showing in the provided grafana dashboards

+1 issue still there

+1 issue still here as well (Kubernetes deployed with Rancher)
# Stack deployed
prometheus-stack                monitoring-ns   10              2022-12-09 12:41:56.026097892 +0100 CET deployed        kube-prometheus-stack-42.2.0            0.60.1

@fguiet, edit the panel and remove name=~"^k8s_.*", from all the panels that aren't showing data. Your current settings are filtering out the view.

same here, kuernetes v1.25.6 installl with kubeadm , kube-prometheu-stack v0.63.0 install with helm

Same here. Number of replicas are correctly reflected in and shown in graph but there N/A for pod cluster usage

Is there any fix for the issue? or any dashboard id out there that I can import. I want to see pod resource uses over time for a deployment.

+1 issue still there

Same problem for me

@fguiet, edit the panel and remove name=~"^k8s_.*", from all the panels that aren't showing data. Your current settings are filtering out the view.

It worked

i all panel that not working ,just edit them and remove all this: "image!="",name=~"^k8s_.*","
it worked for me..


user/references/protocol/rest.md  should be translated to English

http://dubbo.apache.org/en-us/docs/user/references/protocol/rest.html
should be translated to English
Source doc is here ：
https://github.com/apache/dubbo-website/blob/master/docs/en-us/user/references/protocol/rest.md

#368 Hi, questions about translation unified handling here.

OK


Problem building project | Could not resolve androidx.compose.compiler:compiler:1.4.4-dev-k1.8.20-f6ae19e64ff.

FAILURE: Build completed with 8 failures.

1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':androidApp:checkDebugAarMetadata'.
> Could not resolve all files for configuration ':androidApp:debugRuntimeClasspath'.
   > Could not resolve androidx.compose.compiler:compiler:1.4.4-dev-k1.8.20-f6ae19e64ff.
     Required by:
         project :androidApp
      > Could not resolve androidx.compose.compiler:compiler:1.4.4-dev-k1.8.20-f6ae19e64ff.
         > Could not get resource 'https://androidx.dev/storage/compose-compiler/repository/androidx/compose/compiler/compiler/1.4.4-dev-k1.8.20-f6ae19e64ff/compiler-1.4.4-dev-k1.8.20-f6ae19e64ff.pom'.
            > Could not GET 'https://androidx.dev/storage/compose-compiler/repository/androidx/compose/compiler/compiler/1.4.4-dev-k1.8.20-f6ae19e64ff/compiler-1.4.4-dev-k1.8.20-f6ae19e64ff.pom'.
               > Unsupported or unrecognized SSL message

Are you still seeing this?  Can you open https://androidx.dev/storage/compose-compiler/repository/androidx/compose/compiler/compiler/1.4.4-dev-k1.8.20-f6ae19e64ff/compiler-1.4.4-dev-k1.8.20-f6ae19e64ff.pom for example in a browser?

Yes I am able to download in browser

Maybe there's a proxy or something your IDE is using ?

Tried today, and it works fine now without changing anything. Thanks for the responses 🙌


Terminology and ApproxFun

One thing to consider is making things consistent with ApproxFun.jl.  See http://juliaapproximation.github.io/ApproxFun.jl/latest/faq.html#Approximating-functions-1 for example.
If we do, then instead of nodes it should probably be points.
In general, it is a good idea to get a sense of ApproxFun.jl since it is one of the greatest Julia libraries of all.  You may also be able to use it directly for Chebyshev coefficients, if necessary.

Closing this.  I think we should keep it as nodes.