NUMERIC(18,0) 'rounding' to about 16 significant digits

We are using DBeaver to view data in a proprietary database called 'Vortex' (from Trifox.com).
When viewing the relevant column definition in the list of columns, it is listed as 'NUMERIC(18,0)'. It is essentially a 'Long', but the proprietary database engine only reports all numbers as 'NUMERIC(X,Y)'.
When displaying the data in this column, the 18 digits that are stored in it are being binary 'rounded' to a significant number of digits.
For instance, when the column contains the 'raw' value of<PHONE_NUMBER>50676232, it is being displayed as<PHONE_NUMBER>50676220.
I have tried playing around with various setting in the 'Data Formatting' section of the 'Edit Connection' dialog, but nothing seems to 'fix' it.
Any idea how I can get the column displaying all 18 digits of these numbers?

Did you make settings in Connection settings>Data Editor>Data Formats? Please try to change Maximum fraction digits. If it doesn't help, try to disable numeric formatting on the same page

there is no update on issue for a long time.
if it is still actual for you - feel free to reopen it

Sorry, I have been very busy and I hadn't got back to this issue.
The 'Edit connection' dialog on the version of DBeaver that I am running (Version <IP_ADDRESS>006281720) does not have the same fields that you are talking about:

Also, I have tried setting the maximum fraction digits to 18, and this has not fixed the issue.


waterfox: fix download url

#2308 (Outstanding Excavator issues)

https://www.waterfox.net/blog/waterfox-2019.10-release-download/
Classic branch or current version, which is better?

https://www.waterfox.net/blog/waterfox-2019.10-release-download/
Classic branch or current version, which is better?

I tend to add another waterfox-current package for current version, since https://www.waterfox.net/ puts classic version on the download button.
Maybe they think users that have been using Waterfox may be more used to the classic version. (not sure about the details, though)

The files descriptions should end with (classic) and (current), and it would be helpful to document in either the description, notes, or comments, what the differences between the two versions are, IMO.


Investigate failed callback to ARS

This doesn't seem to be a systemic issue (i.e., it doesn't seem to have affected recent standup queries) so it's not a high priority.  But it does appear to be reproducible, so I'm documenting it here...
I posted Query 1 below to the ARS https://ars-prod.transltr.io/ars/api/submit, which resulted in this PK: 79706a52-18b0-4706-a940-e47765f72f45. The ARS query specifically to BTE was given this PK: c5b2340d-ed18-420d-8599-9772020f9990, and I can retrieve the ARS-stored response at https://arax.ncats.io/api/arax/v1.3/response/c5b2340d-ed18-420d-8599-9772020f9990.  That response contains the standard async response, which points to this URL on our prod BTE instance: http://bte.transltr.io/v1/check_query_status/4dS89PiGRx.  The file at that URL is 77MB in size so the query has clearly completed.  At the end of the JSON appears this status: "callback":"Request failed, received code 500"}.
Tentative hypothesis is that file size was too large to be successfully posted to the callback.
Query 1:
{
    "message": {
        "query_graph": {
            "edges": {
                "e0": {
                    "object": "ent",
                    "subject": "gene1"
                },
                "e1": {
                    "object": "gene2",
                    "subject": "gene1"
                },
                "e2": {
                    "object": "gene3",
                    "subject": "gene2"
                },
                "e3": {
                    "object": "sd",
                    "subject": "gene3"
                }
            },
            "nodes": {
                "ent": {
                    "ids": [
                        "MONDO:0024419"
                    ]
                },
                "gene1": {
                    "categories": [
                        "biolink:Gene"
                    ]
                },
                "gene2": {
                    "categories": [
                        "biolink:Gene"
                    ]
                },
                "gene3": {
                    "categories": [
                        "biolink:Gene"
                    ]
                },
                "sd": {
                    "ids": [
                        "MONDO:0006605"
                    ]
                }
            }
        }
    }
}

Mark Williams reports that he thinks the limit is 1 GB.  So file size does not explain the issue here, since the response is just 78 MB. I'm going to close this issue here because it seems to be an issue on the ARS.  If it recurs, I'll create an issue there and link to this one...

I'm reopening this issue since @genomewide noticed an issue that seems to have the same root cause. :

Andy reported that this job stalled https://arax.ncats.io/?r=c5af59b0-2e85-464f-94ff-5f08b4e89c56
The BTE child job is https://arax.ncats.io/?r=d9fff011-064f-42c8-aae9-405200a9cf07
Examining the JSON response, https://arax.ncats.io/api/arax/v1.3/response/d9fff011-064f-42c8-aae9-405200a9cf07 points to https://bte.transltr.io/v1/check_query_status/8vZBnysTz1, which reports the status as completed (results expired)  (When I looked at the link yesterday, a valid TRAPI was returning, so looks like the cache expired. I think this is normal behavior and unrelated to the primary issue here.)
The BTE response also reports Request failed, received code 500, which I think refers to an error that BTE had posting to ARS, which is why the ARS results were never updated

Tagging @markdwilliams and @tokebe if they have thoughts on the possible issue and strategies to diagnose...

I can confirm that the 500 message means it encountered that while attempting to post to the callback URL, I can also confirm that it should just be the cache expiring as the reason why it can no longer be retrieved.
I’m not aware of what might cause a 500 response from ARAX. I’ll definitely check tomorrow if there’s anything that generates a 500 from BTE, just in case.

My previous query was odd with a large set of curries. With a new query it failed for me again.
https://arax.ncats.io/?r=b2e7c0b9-5eb3-4f9f-b89d-b534280d14b1
This was the query that worked for CDK1 and CDK2 but failed at CDK3. I waited for each to finish before posting the next.
{   "edges": {     "e0": {       "object": "n0",       "predicates": [         "biolink:affects"       ],       "subject": "n1"     },     "e1": {       "object": "n1",       "predicates": [         "biolink:affects"       ],       "subject": "n2"     }   },   "nodes": {     "n0": {       "categories": [         "biolink:Protein"       ],       "ids": [         "UniProtKB:P11802"       ],       "is_set": false,       "name": "CDK4"     },     "n1": {       "categories": [         "biolink:Gene",         "biolink:Protein"       ],       "is_set": false     },     "n2": {       "categories": [         "biolink:ChemicalEntity",         "biolink:Drug",         "biolink:SmallMolecule"       ],       "is_set": false     }   } }

Per the slack thread here, I'm going to close this issue as hopefully resolved...


[express] Make ExpressAdapter fields protected instead of private

Private fields make it difficult to extends ExpressAdapter. Using protected instead make ExpressAdapter open to extension.
Close #626

thank you for this PR
released in v5.8.3


Nunjucks?

Ref this: http://stackoverflow.com/questions/31514005/render-precompilled-nunjucks-browserifynunjucksify-with-ampersand-view
There is a clash as both ampersand-view and nunjucks (precompiled) want their own this on the render-function?
If this is the case, any workarounds?
~Cheers~

Disregard this issue, I'm leaving just in case others have the same issue.
The problem is non-existent, as nunjucks needs to render the template before we set/pass it  to the view. This way the first rendering will fill in all non-model (static) props, such as translations etc, and then the view handles all model-props.


lxml code completion

I am trying to use lxml package to parse XML file.
Simple code work well in python interpreter.
But,  PTVS code completion does not show anything about package.
from lxml import etree
etree.parse(path)

Is this a usual things? or may I miss something to import package?
Versions:
Windows 10 & VS2017 & Anaconda 4.4.0 & Python 3.6 & lxml 3.8.0

Thanks. I can't confirm it at the moment but I can test again later when my development build isn't so broken.
You can check the Python Environments window to see if the database is up to date. If there's a checkmark next to lxml, and it doesn't work, then there's probably nothing you can do.
Are you on VS 2017 15.3?

Note: that etree comes from a .pyd, here's the file list for lxml:
__init__.py
__pycache__
_elementpath.py
builder.py
cssselect.py
doctestcompare.py
ElementInclude.py
etree.cp36-win32.pyd
html
includes
isoschematron
lxml.etree.h
lxml.etree_api.h
objectify.cp36-win32.pyd
pyclasslookup.py
sax.py
usedoctest.py

My Current Visual Studio Version is
Microsoft Visual Studio Community 2017
Version 15.3.2
Python   3.0.17208.1
lxml package is recognized well to database. I already try Refresh DB

Here is Continuum\Anaconda3\Lib\site-packages\lxml folder

I think I figure it out. The problem was environment settings.
** Python 3.6 & lxml 3.8.0 ** does not show and code completion for C based python package.
After I change to ** Python 3.5.3 & lxml 3.7.3 **, code completion now works well.
Thanks @huguesv for replying my issue!

I wasn't able to get completions on lxml.etree, on either 3.5 or 3.6 venvs. I also tried an lxml 3.7.3.

Closing as duplicate of https://github.com/Microsoft/PTVS/issues/3046


[Question] How to generate junit.xml without BrowserType in name

Hi,
How can I generate JUnit .xml with the same name as test case, without BrowserType or parameter in name.
e.g.
actual:

expected:

Motivation:
Our test management tool based on date from .xml, so we need the same format in .xml as required to execute test case.
e.g.
classname="test_example"
name="test_has_title"
CI/CD will use this as parameters:
pytest -v {classname}.py::{name}

We internally parameterise as well: https://github.com/microsoft/playwright-pytest/blob/19659038de3a7d22b55485ddc7535e253881bcef/pytest_playwright/pytest_playwright.py#L88-L91
not sure if we can do much there, Pytest has param id which would allow you to change it but probably it needs to be unique:
https://docs.pytest.org/en/stable/reference/reference.html#pytest-param


video element live stream

I want to use video element with  live stream for webinar software how can I set manifesto for this?

What technology do you apply to use? Did you checked with webrtc supported?
For example, go to https://test.webrtc.org in both chrome and your webview in devices (of course, enable needed permissions for webrtc work properly).

Thanks, @dphans!
Good point, as using that link probably is the easiest way to debug (and develop) this.

Thanks for answers @dphans


Allow additional properties in global

Describe your changes :
Because the openmetadata chart is using the special global values and is not allowing any additional properties in the json schema this chart can effectively not be used as a dependency of a chart that also uses these special global values. The correct way to resolve this is to not use global and I see that an effort is being made in #143. Allowing additional values is an intermediate, non-breaking step to allow this chart to be used as a dependency (at the cost of not checking for additional properties within the first level of global).

Type of change :

[x] Bug fix

Checklist:

[x] I have read the CONTRIBUTING document.
[x] I have performed a self-review of my own.
[x] I have tagged my reviewers below.
[x] My changes generate no new warnings.
[x] All new and existing tests passed.

Reviewers

@akash-jain-10

Hi guys, just let you know seems the issue is still there for the 1.0.2, not sure if i was wrong
https://openmetadata.slack.com/archives/C02B6955S4S/p1685000159089829?thread_ts=1684835342.380029&cid=C02B6955S4S


admin interface for editing weightings for services and quotes

need a way of viewing and adjusting the weightings for each service / quote pair

closed with pr #53


expand macros with argument lists

followup to #25, now we need to make function-like macros that can take arguments for their expansions (since these are the actually interesting macros).
This will require changes to the lexing and parsing structure, probably.

being handled in #35


npm install found 17 vulnerabilities (8 low, 6 moderate, 3 high)

Issue description
Upon reaching step 5: Install Dependencies, a warning was produced:
found 17 vulnerabilities (8 low, 6 moderate, 3 high)
https://docs.origintrail.io/dkg-v6-upcoming-version/setup-instructions-dockerless#step-5-install-dependencies
Expected behavior
Understand that this may be quite normal (https://www.voitanos.io/blog/don-t-be-alarmed-by-vulnerabilities-after-running-npm-install/)
Nonetheless may be scary to new node runners. May be good to add a short description of why this happens in the docs.
Steps to reproduce the problem

Follow the tutorial https://docs.origintrail.io/dkg-v6-upcoming-version/setup-instructions-dockerless#step-5-install-dependencies
Warning occurs at Step 5: Install Dependencies

Specifications

Node version: v6 ot-node (test node)
Platform: Ubuntu 20.04.3 LTS

Error logs
Upon checking 'npm audit', the 3 'high' issues are:

node-fetch is vulnerable to Exposure of Sensitive Information to an Unauthorized Actor

Exposure of Sensitive Information in simple-get

Exposure of Sensitive Information in simple-get

Closing as this is clearer in the instructions now. Thanks @Valcyclovir


Fix wrongly indented bullet point

Just a small typo/error I found while reading through the list.

Thanks!


frontend: don't disable back-button in send

Don't disable the back button in send.tsx when a dialog/modal (QR scanner, or UTXO selection) is open. It is a UX best practice to do this when a modal is open however a) our dialog CSS creates an overlay which means nothing behind it can be clicked anyways, and b) the back button currently supports ESC which should close the modal but navigates back from the send route right now.
If we don't need the is-open state for these modals we can simplify the send component in another refactor by extracting coin control into its own sub-component (moving activeCoinControl there) and moving activeScanQR into ReceiverAddressInput.
This PR does not change anything because the back-button is not clickable because of the screen size (100%) overlay used:
https://github.com/BitBoxSwiss/bitbox-wallet-app/blob/bdf270c0bfc824f3dfda91176f76852eb4a524f7/frontends/web/src/components/dialog/dialog.module.css#L9-L12

having this as a PR is a bit silly, I'll do it in #2871 and #2872


bug: garaga declare cli command gives false positive error.

see https://github.com/keep-starknet-strange/garaga/discussions/217

I have a snarkjs generated artifacts (verification_key.json, public_inputs.json, proof.json, etc).
I generate cairo contract using garaga gen --system groth16 --vk verification_key.json, which created 3 files, as expected, but when I trying to declare project (garaga declare-project --project-path my/project/path --env-file .secrets --network sepolia --fee strk), i get an error: class hash not found - deploying Error during declaration: Client failed with code 20. Message: Contract not found.
Im sure that path to project is correct. I didnt changed autogenerated cairo files too.
Thank you in advance.

there is an issue in the flow here, when the class hash doesn't exists it still declares it but an error is raised in python.
https://github.com/keep-starknet-strange/garaga/blob/47efa2272528c966535c75be7ac19042f8161ab0/hydra/garaga/starknet/cli/smart_contract_project.py#L72
https://github.com/keep-starknet-strange/garaga/blob/47efa2272528c966535c75be7ac19042f8161ab0/hydra/garaga/starknet/cli/declare.py#L93
it seems the error is wrongly raised up to the calling function despite being caught :thinking:
When running the command for a 2nd time, it correctly detects that the class hash exists and prints it.

Can i take this up?


💄(react) change label color when focused ( select, input )

We forgot to implement this behavior when coding these inputs.

About Figma, I think there is a mistake about icon color (Should we apply color to icon for error and focus state ?) I wrote a comment to about that, let's see. At first, I said we do not change icon color.

Ditto, for disabled state, the label color change.

Ditto, for disabled state, the label color change.

I do not see a change about that

Ditto, for disabled state, the label color change.

I do not see a change about that

Ditto, for disabled state, the label color change.

I do not see a change about that

Arf, CSS priority issue.


Extend SharePoint docs on ALM APIs using the CLI

Submit PR to SharePoint docs at https://docs.microsoft.com/en-us/sharepoint/dev/apis/alm-api-for-spfx-add-ins describing how to manage lifecycle of apps using the Office 365 CLI.

I'll close the issue once it's been merged, ok?

Docs PR https://github.com/SharePoint/sp-dev-docs/pull/1301 has been merged so closing this one. Thanks @andrewconnell!


PackageId:CPython39.Exe.x64;PackageAction:DownloadPackage;ReturnCode:0x80072ee2;

This issue has been moved from a ticket on Developer Community.

在此处描述你的问题
===DO NOT EDIT BELOW THIS LINE===
PackageId:CPython39.Exe.x64;PackageAction:DownloadPackage;ReturnCode:0x80072ee2;

Original Comments
Feedback Bot on 12/6/2022, 06:31 PM:
(private comment, text removed)

Original Solutions
(no solutions)

Hi there, please see https://learn.microsoft.com/en-us/visualstudio/python/installing-python-support-in-visual-studio?view=vs-2022#troubleshooting and reopen the ticket if it doesn't resolve your issue. Thanks!


Yet we don't break apis, this patch reverts the next:

https://github.com/thorvg/thorvg/commit/cb36c25aff0d779018917b8e83e1fa386112e4e1
https://github.com/thorvg/thorvg/commit/8c0fc2b25a1d804d8c2ea32fc8046eeafc0f7dcd
https://github.com/thorvg/thorvg/commit/c1e4e0808ac46a63c3e7067e3490285e3be89160
https://github.com/thorvg/thorvg/commit/899ea7769575bb973975200227c11d4cf147e6bc
Those reverted changes sholud be in the bucket, the next issue item:
@issue: https://github.com/thorvg/thorvg/issues/1372

@mgrudzinska  the offset logic applied, this only reverts the api breaks. I expected the deprecate the previous ones and beta apis for the new ones.
You can submit additional changes for the stroke-offset apis like this pending change:
https://github.com/thorvg/thorvg/pull/1564
We will apply them when 1.0 is get ready.
p.s offset feature is buggy im looking into that.


Move kubernetes-charts location

Currently, stable charts are pointed at https://storage.googleapis.com/kubernetes-charts. This puts it on a sub-path. Instead, can we move it to https://kubernetes-charts.storage.googleapis.com. This is top level and works today.
It would also ease the transition to a custom kubernetes domain.

I put a PR up at #1545.

FYI, docs at https://cloud.google.com/storage/docs/xml-api/reference-uris

cc/ @viglesiasce @prydonius

Note, I'm going to make another change for the endpoint to be https:// instead of http://.

#1545 has been updated. Ready for reviewer eyes.


Changed SIGINT back to 2

The use of SIGINT adds the dependency of the PCNTL extension
Since this worked before with the integer, there's no need to add the
requirement of the PCNTL extension for a single constant.

Might be nice to have a code comment to note what signal "2" is.

@boone Sounds good. I'll add a comment 😄


Add new ParsePush.jar

Build and upload ParsePush.jar that's generated locally from internal Parse repo.
Fixes #12

Awesome! Tested and working on Unity for Android. Thanks!

Any reason we can't embed the source of ParsePush/scripts to generate instead of just the Jar? We should allow contributions back here, no?

Currently we can't.

ParsePush is part of our internal repo currently. We haven't open-source it
Even if it's open-sourced, it won't be part of Parse-SDK-dotNET. Ideally it will be a separate repo.
The script to generate the .jar is tied to our internal Android SDK repo.

If we eventually manage to open-source ParsePush, it will be in a form of subclone/submodule. After that then we can add the script to this repo to build ParsePush.jar.

Closing this and doing #75 instead.


Vulnerability roundup 107: bento4-1.6.0-637: 1 advisory [8.8]

search, files

[ ] CVE-2021-32265 CVSSv3=8.8 (nixos-21.05)

CVE details
CVE-2021-32265
An issue was discovered in Bento4 through v1.6.0-637. A global-buffer-overflow exists in the function AP4_MemoryByteStream::WritePartial() located in Ap4ByteStream.cpp. It allows an attacker to cause code execution or information disclosure.

Scanned versions: nixos-21.05: 3b1789322fc.
Cc @makefu

NixOS 21.05 is EOL. Update is in 21.11 and unstable.


Bump: 0.3.0-next -> 0.4.0

Proposed changes
This PR bumps FIWARE Big Bang version from 0.3.0-next -> 0.4.0.
Types of changes
What types of changes does your code introduce to the project: Put an x in the boxes that apply

[ ] Bugfix (non-breaking change which fixes an issue)
[X] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] Update only documentation, not any source code.

Checklist
Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of
them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before
merging your code.

[X] I have read the CONTRIBUTING doc
[ ] I have signed the CLA
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] I have added necessary documentation (if appropriate)
[ ] Any dependent changes have been merged and published in downstream modules

Further comments
N/A

Codecov Report

Merging #67 (4a77ab4) into main (f33d91f) will not change coverage.
The diff coverage is 100.00%.

@@           Coverage Diff           @@
##             main      #67   +/-   ##
=======================================
  Coverage   90.21%   90.21%
=======================================
  Files           2        2
  Lines         797      797
=======================================
  Hits          719      719
  Misses         78       78

Impacted Files
Coverage Δ

lets-fiware.sh
89.51% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f33d91f...4a77ab4. Read the comment docs.


api/worker, koji: fix race condition when using multiple listeners

When remote worker socket was enabled, this was happening:
e := echo.New()

go func() {
    e.Listener = listener1
    e.Start("")
}()

e.Listener = listener2
e.Start("")

Yeah, this is a race condition. None of the echo's Start methods cannot safely
handle multiple listeners.
This commit fixes this issue by using Echo only as a router for standard
http.Server which handles multiple listeners in a non-racy way.

#997, otherwise good.


NinePatchRects do not display their Textures after certain circumstances

Godot version:

v3.2.3.stable.mono.official
OS/device including version:

Windows 10 20H2
1030 GT, not sure on driver version
GLES3 backend
Issue description:

This is a convoluted one.
I am creating NinePatchRects through code, and assigning them Textures based upon their parent's "Element Name". I use a data-driven UI skinning system, so this is why.
I am also loading all of my Textures from file at the start of the game, and distributing them to the skinnable UI components from a central space.
I load PackedScenes representing each state of my game, and add their Instance() to a Control node I call "MainUI".
When I load in a new PackedScene this way, I free the old Instance() node, and remove it from the tree.
This is where it gets strange.
On the second (and subsequent) time a PackedScene that contains the GUI skinning component scripts are loaded in, and skinned, the NinePatchRects do not display their texture. They are visible, properly sized, and modulated with a non-zero alpha colour.
Even stranger, if I do not free the previous screen, the NinePatchRects of the previous screen do not render their Textures, once the new screen is in place, and are not rendering their Textures, either.
Because of another issue, it is not possible to check if the NinePatchRects have any Textures in the Remote Tree view, but it can be verified with a debugger.
I hope this has a simple fix, and it's something I'm doing wrong!
Steps to reproduce:

Run the test project
Notice that the first screen has a background, and various UI elements. The pink/black square is not part of the UI skinning system.
Push Space.
Notice that the new screen that is instantiated, has NinePatchRects, created in the same way, but their Textures do not render.
Click the "New Game" 'button'
Notice that it goes back to the previous screen, and there are no Textures being displayed.

For bonus points:

Go to LoadingState.cs, and switch the commenting on the GetNextState() function
Run the project
Notice that this screen has properly displaying Textures in the NinePatchRects.
Click "New Game".
Notice that this screen has no Textures displaying in the NinePatchRects. Again, the black/pink square is not part of the skinnable UI system.
Repeat the same process as the last section, and note that no Textures are displayed.

Minimal reproduction project:

JoyGodot - Trimmed.zip

Closing this issue because the minimum reproduction project is massive, and I can't seem to trim it down and still reproduce the bug.


cni: autodetect bin dir

I don't think there is an API we can do to fetch this, but we can definitely add a map of GKE -> /xyz, Openshift -> /abc into the install flow to make it a bit simpler to get started
cc @bianpengyuan

In particular without this we cannot really run the integ tests on GKE easily

By installation time, you mean at istioctl? Do we have a reliable way to tell platform there? Should we instead do it at CNI directly?

You can tell gke from the version number (in istioctl - maybe even helm). Not sure a better way unfortunately. It would be cool if they had an API to expose the CNI settings but I don;t think they do

Could you remind me why we cannot run tests on GKE easily without this? I think we can pass in some overlay in the test job?

We probably can run it on GKE (I am sure we do internally?), I couldn't figure out how though. I wouldn't consider this a high priority, just nice to have, especially if CNI becomes more common its nice to not have churn on install


poor bandwidth on dual Radeon Pro VII GPUs

Hello everyone
We recently added a second Radeon Pro VII to our simulation system. Unfortunately, though, it seems the GPUs do not want to talk to each other, although they are directly connected with an Infinity Fabric Link Bridge.
The system usually runs Arch Linux, where I also started a discussion about the issue, but testing with Ubuntu shows the same issue. Everything posted here was done on the Ubuntu system.
system
hardware setup

GPUs: 2 AMD Radeon Pro VII
CPU: AMD Ryzen Threadripper 2950X
mainboard: Asus X399-A

The GPUs are connected with an Infinity Fabric Link Bridge.
software

OS: Ubuntu 20.04.3
kernel: 5.11
ROCM: installed via sudo amdgpu-install --usecase=rocm with amdgpu-install from here

other requirements
I did verify that critical requirements according to the ROCM supported hardware page are met, eg. hardware (see above), but also
IOMMU
$ sudo dmesg | grep -i iommu
[sudo] password for tinux:
[    0.271162] iommu: Default domain type: Translated
[    0.471020] pci 0000:00:00.2: AMD-Vi: IOMMU performance counters supported
[    0.471076] pci 0000:40:00.2: AMD-Vi: IOMMU performance counters supported
[    0.471120] pci 0000:00:01.0: Adding to iommu group 0
[    0.471133] pci 0000:00:01.1: Adding to iommu group 1
[    0.471146] pci 0000:00:01.2: Adding to iommu group 2
[    0.471166] pci 0000:00:02.0: Adding to iommu group 3
[    0.471183] pci 0000:00:03.0: Adding to iommu group 4
[    0.471195] pci 0000:00:03.1: Adding to iommu group 5
[    0.471213] pci 0000:00:04.0: Adding to iommu group 6
[    0.471231] pci 0000:00:07.0: Adding to iommu group 7
[    0.471243] pci 0000:00:07.1: Adding to iommu group 8
[    0.471261] pci 0000:00:08.0: Adding to iommu group 9
[    0.471273] pci 0000:00:08.1: Adding to iommu group 10
[    0.471297] pci 0000:00:14.0: Adding to iommu group 11
[    0.471308] pci 0000:00:14.3: Adding to iommu group 11
[    0.471368] pci 0000:00:18.0: Adding to iommu group 12
[    0.471379] pci 0000:00:18.1: Adding to iommu group 12
[    0.471390] pci 0000:00:18.2: Adding to iommu group 12
[    0.471401] pci 0000:00:18.3: Adding to iommu group 12
[    0.471414] pci 0000:00:18.4: Adding to iommu group 12
[    0.471425] pci 0000:00:18.5: Adding to iommu group 12
[    0.471436] pci 0000:00:18.6: Adding to iommu group 12
[    0.471447] pci 0000:00:18.7: Adding to iommu group 12
[    0.471506] pci 0000:00:19.0: Adding to iommu group 13
[    0.471517] pci 0000:00:19.1: Adding to iommu group 13
[    0.471529] pci 0000:00:19.2: Adding to iommu group 13
[    0.471542] pci 0000:00:19.3: Adding to iommu group 13
[    0.471553] pci 0000:00:19.4: Adding to iommu group 13
[    0.471565] pci 0000:00:19.5: Adding to iommu group 13
[    0.471577] pci 0000:00:19.6: Adding to iommu group 13
[    0.471588] pci 0000:00:19.7: Adding to iommu group 13
[    0.471622] pci 0000:01:00.0: Adding to iommu group 14
[    0.471635] pci 0000:01:00.1: Adding to iommu group 14
[    0.471649] pci 0000:01:00.2: Adding to iommu group 14
[    0.471654] pci 0000:02:00.0: Adding to iommu group 14
[    0.471658] pci 0000:02:01.0: Adding to iommu group 14
[    0.471662] pci 0000:02:02.0: Adding to iommu group 14
[    0.471666] pci 0000:02:03.0: Adding to iommu group 14
[    0.471670] pci 0000:02:04.0: Adding to iommu group 14
[    0.471674] pci 0000:02:09.0: Adding to iommu group 14
[    0.471678] pci 0000:05:00.0: Adding to iommu group 14
[    0.471683] pci 0000:08:00.0: Adding to iommu group 14
[    0.471695] pci 0000:09:00.0: Adding to iommu group 15
[    0.471707] pci 0000:0a:00.0: Adding to iommu group 16
[    0.471719] pci 0000:0b:00.0: Adding to iommu group 17
[    0.471744] pci 0000:0c:00.0: Adding to iommu group 18
[    0.471759] pci 0000:0c:00.1: Adding to iommu group 19
[    0.471772] pci 0000:0d:00.0: Adding to iommu group 20
[    0.471784] pci 0000:0d:00.2: Adding to iommu group 21
[    0.471798] pci 0000:0d:00.3: Adding to iommu group 22
[    0.471810] pci 0000:0e:00.0: Adding to iommu group 23
[    0.471825] pci 0000:0e:00.2: Adding to iommu group 24
[    0.471838] pci 0000:0e:00.3: Adding to iommu group 25
[    0.471856] pci 0000:40:01.0: Adding to iommu group 26
[    0.471872] pci 0000:40:02.0: Adding to iommu group 27
[    0.471890] pci 0000:40:03.0: Adding to iommu group 28
[    0.471902] pci 0000:40:03.1: Adding to iommu group 29
[    0.471920] pci 0000:40:04.0: Adding to iommu group 30
[    0.471937] pci 0000:40:07.0: Adding to iommu group 31
[    0.471949] pci 0000:40:07.1: Adding to iommu group 32
[    0.471968] pci 0000:40:08.0: Adding to iommu group 33
[    0.471981] pci 0000:40:08.1: Adding to iommu group 34
[    0.471994] pci 0000:41:00.0: Adding to iommu group 35
[    0.472006] pci 0000:42:00.0: Adding to iommu group 36
[    0.472031] pci 0000:43:00.0: Adding to iommu group 37
[    0.472048] pci 0000:43:00.1: Adding to iommu group 38
[    0.472061] pci 0000:44:00.0: Adding to iommu group 39
[    0.472074] pci 0000:44:00.2: Adding to iommu group 40
[    0.472086] pci 0000:44:00.3: Adding to iommu group 41
[    0.472100] pci 0000:45:00.0: Adding to iommu group 42
[    0.472113] pci 0000:45:00.2: Adding to iommu group 43
[    0.502585] pci 0000:00:00.2: AMD-Vi: Found IOMMU cap 0x40
[    0.502595] pci 0000:40:00.2: AMD-Vi: Found IOMMU cap 0x40
[    0.503499] perf/amd_iommu: Detected AMD IOMMU #0 (2 banks, 4 counters/bank).
[    0.503517] perf/amd_iommu: Detected AMD IOMMU #1 (2 banks, 4 counters/bank).
[    1.017979] AMD-Vi: AMD IOMMUv2 driver by Joerg Roedel<EMAIL_ADDRESS>
CRAT
$ sudo dmesg | grep -i crat
[    0.000000] ACPI: CRAT 0x0000000077CDE878 001DF8 (v01 AMD    AMD CRAT 00000001 AMD  00000001)
[    0.000000] ACPI: Reserving CRAT table memory at [mem 0x77cde878-0x77ce066f]
[    1.168518] amdgpu: Ignoring ACPI CRAT on non-APU system
[    1.168521] amdgpu: Virtual CRAT table created for CPU
[    2.265620] amdgpu: Virtual CRAT table created for GPU
[    3.261272] amdgpu: Virtual CRAT table created for GPU

Atomics
$ sudo dmesg | grep -i kfd
[    2.177738] kfd kfd: amdgpu: Allocated 3969056 bytes on gart
[    2.265959] kfd kfd: amdgpu: added device 1002:66a1
[    3.169496] kfd kfd: amdgpu: Allocated 3969056 bytes on gart
[    3.261636] kfd kfd: amdgpu: added device 1002:66a1

and
$ sudo lspci -vvv -s 43:00.0 | grep Atomic
			 AtomicOpsCap: 32bit+ 64bit+ 128bitCAS-
			 AtomicOpsCtl: ReqEn+

issues
It seems the GPUs are not connected to each other, despite the fact that they are physically connected with an Infinity Fabric Link Bridge.
tests with rocm-smi
$ sudo rocm-smi --shownodesbw

======================= ROCm System Management Interface =======================
================================== Bandwidth ===================================
       GPU0         GPU1
GPU0   N/A          0-0
GPU1   0-0          N/A
Format: min-max; Units: mps
"0-0" min-max bandwidth indicates devices are not connected dirrectly
============================= End of ROCm SMI Log ==============================

I also ran a few other test, but I cannot really make sense of it, given the output of the command above.
$ sudo rocm-smi --showtopoaccess

======================= ROCm System Management Interface =======================
===================== Link accessibility between two GPUs ======================
       GPU0         GPU1
GPU0   True         True
GPU1   True         True
============================= End of ROCm SMI Log ==============================

and
$ sudo rocm-smi --showtopo

======================= ROCm System Management Interface =======================
=========================== Weight between two GPUs ============================
       GPU0         GPU1
GPU0   0            15
GPU1   15           0

============================ Hops between two GPUs =============================
       GPU0         GPU1
GPU0   0            1
GPU1   1            0

========================== Link Type between two GPUs ==========================
       GPU0         GPU1
GPU0   0            XGMI
GPU1   XGMI         0

================================== Numa Nodes ==================================
GPU[0]		: (Topology) Numa Node: 0
GPU[0]		: (Topology) Numa Affinity:<PHONE_NUMBER>
GPU[1]		: (Topology) Numa Node: 0
GPU[1]		: (Topology) Numa Affinity:<PHONE_NUMBER>
============================= End of ROCm SMI Log ==============================

other benchmarks
I also ran a benchmark from the RCCL repository, which is much slower on 2 GPUs than on a single.
2 GPUs
$ sudo ./build/all_reduce_perf -b 8 -e 128M -f 2 -g 2
# nThreads: 1 nGpus: 2 nRanks: 1 minBytes: 8 maxBytes: 134217728 step: 2(factor) warmupIters: 5 iters: 20 validation: 1
#
# Using devices
#   Rank  0 Pid   2916 on  ultrafast device  0 [0000:0c:00.0] AMD Radeon (TM) Pro VII
#   Rank  1 Pid   2916 on  ultrafast device  1 [0000:43:00.0] AMD Radeon (TM) Pro VII
#
#                                                       out-of-place                       in-place
#       size         count      type   redop     time   algbw   busbw  error     time   algbw   busbw  error
#        (B)    (elements)                       (us)  (GB/s)  (GB/s)            (us)  (GB/s)  (GB/s)
           8             2     float     sum    24.85    0.00    0.00  0e+00    21.89    0.00    0.00  0e+00
          16             4     float     sum    20.07    0.00    0.00  0e+00    19.69    0.00    0.00  0e+00
          32             8     float     sum    19.91    0.00    0.00  0e+00    19.60    0.00    0.00  0e+00
          64            16     float     sum    19.61    0.00    0.00  0e+00    19.63    0.00    0.00  0e+00
         128            32     float     sum    19.78    0.01    0.01  0e+00    21.51    0.01    0.01  0e+00
         256            64     float     sum    19.76    0.01    0.01  0e+00    19.83    0.01    0.01  0e+00
         512           128     float     sum    19.98    0.03    0.03  0e+00    19.97    0.03    0.03  0e+00
        1024           256     float     sum    35.68    0.03    0.03  0e+00    35.36    0.03    0.03  0e+00
        2048           512     float     sum    20.42    0.10    0.10  0e+00    20.13    0.10    0.10  0e+00
        4096          1024     float     sum    37.20    0.11    0.11  0e+00    37.01    0.11    0.11  0e+00
        8192          2048     float     sum    36.14    0.23    0.23  0e+00    33.72    0.24    0.24  0e+00
       16384          4096     float     sum    33.62    0.49    0.49  0e+00    32.19    0.51    0.51  0e+00
       32768          8192     float     sum    32.93    1.00    1.00  0e+00    32.84    1.00    1.00  0e+00
       65536         16384     float     sum    34.00    1.93    1.93  0e+00    33.47    1.96    1.96  0e+00
      131072         32768     float     sum    35.17    3.73    3.73  0e+00    34.86    3.76    3.76  0e+00
      262144         65536     float     sum    38.97    6.73    6.73  0e+00    38.77    6.76    6.76  0e+00
      524288        131072     float     sum    49.84   10.52   10.52  0e+00    49.69   10.55   10.55  0e+00
     1048576        262144     float     sum    66.13   15.86   15.86  0e+00    65.54   16.00   16.00  0e+00
     2097152        524288     float     sum    97.07   21.61   21.61  0e+00    97.34   21.55   21.55  0e+00
     4194304       1048576     float     sum    160.2   26.19   26.19  0e+00    160.3   26.16   26.16  0e+00
     8388608       2097152     float     sum    284.9   29.45   29.45  0e+00    285.0   29.43   29.43  0e+00
    16777216       4194304     float     sum    532.9   31.48   31.48  0e+00    536.1   31.30   31.30  0e+00
    33554432       8388608     float     sum   1043.1   32.17   32.17  0e+00   1056.0   31.77   31.77  0e+00
    67108864      16777216     float     sum   2072.9   32.37   32.37  0e+00   2074.7   32.35   32.35  0e+00
   134217728      33554432     float     sum   4095.4   32.77   32.77  0e+00   4096.3   32.77   32.77  0e+00
# Errors with asterisks indicate errors that have exceeded the maximum threshold.
# Out of bounds values : 0 OK
# Avg bus bandwidth    : 9.86367
#

1 GPU
$ sudo ./build/all_reduce_perf -b 8 -e 128M -f 2 -g 1
# nThreads: 1 nGpus: 1 nRanks: 1 minBytes: 8 maxBytes: 134217728 step: 2(factor) warmupIters: 5 iters: 20 validation: 1
#
# Using devices
#   Rank  0 Pid   3122 on  ultrafast device  0 [0000:0c:00.0] AMD Radeon (TM) Pro VII
#
#                                                       out-of-place                       in-place
#       size         count      type   redop     time   algbw   busbw  error     time   algbw   busbw  error
#        (B)    (elements)                       (us)  (GB/s)  (GB/s)            (us)  (GB/s)  (GB/s)
           8             2     float     sum     8.38    0.00    0.00  0e+00     4.89    0.00    0.00  0e+00
          16             4     float     sum     7.77    0.00    0.00  0e+00     5.24    0.00    0.00  0e+00
          32             8     float     sum     7.27    0.00    0.00  0e+00     8.03    0.00    0.00  0e+00
          64            16     float     sum     7.46    0.01    0.00  0e+00     3.99    0.02    0.00  0e+00
         128            32     float     sum     8.42    0.02    0.00  0e+00     3.82    0.03    0.00  0e+00
         256            64     float     sum     7.72    0.03    0.00  0e+00     4.25    0.06    0.00  0e+00
         512           128     float     sum     8.05    0.06    0.00  0e+00     4.40    0.12    0.00  0e+00
        1024           256     float     sum     7.66    0.13    0.00  0e+00     4.01    0.26    0.00  0e+00
        2048           512     float     sum     9.16    0.22    0.00  0e+00     4.56    0.45    0.00  0e+00
        4096          1024     float     sum     7.51    0.55    0.00  0e+00     4.10    1.00    0.00  0e+00
        8192          2048     float     sum     7.88    1.04    0.00  0e+00     3.92    2.09    0.00  0e+00
       16384          4096     float     sum     7.84    2.09    0.00  0e+00     3.71    4.41    0.00  0e+00
       32768          8192     float     sum     7.42    4.42    0.00  0e+00     3.80    8.63    0.00  0e+00
       65536         16384     float     sum     7.45    8.80    0.00  0e+00     4.27   15.35    0.00  0e+00
      131072         32768     float     sum     8.17   16.05    0.00  0e+00     4.47   29.31    0.00  0e+00
      262144         65536     float     sum     9.10   28.81    0.00  0e+00     3.71   70.69    0.00  0e+00
      524288        131072     float     sum    39.66   13.22    0.00  0e+00     3.69  142.27    0.00  0e+00
     1048576        262144     float     sum    12.87   81.45    0.00  0e+00     3.96  264.85    0.00  0e+00
     2097152        524288     float     sum    14.53  144.29    0.00  0e+00     2.92  718.67    0.00  0e+00
     4194304       1048576     float     sum    23.76  176.54    0.00  0e+00     3.21  1308.00    0.00  0e+00
     8388608       2097152     float     sum    36.37  230.62    0.00  0e+00     3.60  2330.23    0.00  0e+00
    16777216       4194304     float     sum    67.07  250.16    0.00  0e+00     3.30  5079.62    0.00  0e+00
    33554432       8388608     float     sum    123.2  272.40    0.00  0e+00     3.19  10509.90    0.00  0e+00
    67108864      16777216     float     sum    240.4  279.14    0.00  0e+00     3.18  21079.55    0.00  0e+00
   134217728      33554432     float     sum    470.8  285.08    0.00  0e+00     4.88  27490.11    0.00  0e+00
# Errors with asterisks indicate errors that have exceeded the maximum threshold.
# Out of bounds values : 0 OK
# Avg bus bandwidth    : 0
#

Any help is highly appreciated.

@t1nux Apologies for the lack of response. Please check if issue still occurs with the latest ROCm 6.2. If not, please close the ticket. Thanks!

@t1nux Apologies for the lack of response. Please check if issue still occurs with the latest ROCm 6.2. If not, please close the ticket. Thanks!

I will try to do this in the next days, after updating to 6.2. The issue currently still exists on 6.0.

Hi @t1nux,
Looking at your logs, there doesn't seem to be anything obviously wrong, however for rocm-smi --shownodebw, the output does seem to indicate that they aren't connected. This may be a bug in rocm-smi or something wrong with the connection. For the rccl-tests that you ran, the single GPU results are not valid, as a single GPU RCCL operation would just be a memory copy, and so you cannot gauge speed based on those results. Those tests were designed for multi-GPU setups.
Could you please try running the bandwidth tests? ROCm Bandwidth Tests. This should give me a more accurate picture of the bandwidth between the two GPUs. Also, could you please verify that the bridge has 2 lights illuminated corresponding to the number of GPU's that are connected to it? Another user was facing an issue with an improper connection here: Bridge Connection Issue. Please give those a try and let me know if you run into any issues, thanks!

Hello @darren-amd
Apologies that this took so long... and thank you for your patience!!
Here are the results that the ROCm Bandwidth Test gave me.
Without knowing much about what I can actually expect here, based on this, I would even say this looks ok. However, I would very much appreciate a professional opinion on these bandwidth values.
./rocm-bandwidth-test
................................................................
          RocmBandwidthTest Version: 2.6.0

          Launch Command is: ./rocm-bandwidth-test (rocm_bandwidth -a + rocm_bandwidth -A)

          Device: 0,  AMD Ryzen Threadripper 2950X 16-Core Processor
          Device: 1,  AMD Radeon Pro VII,  GPU-ff02618173c6451f,  0c:0.0
          Device: 2,  AMD Radeon Pro VII,  GPU-25e0596172e17d3e,  43:0.0

          Inter-Device Access

          D/D       0         1         2

          0         1         1         1

          1         1         1         1

          2         1         1         1

          Inter-Device Numa Distance

          D/D       0         1         2

          0         0         20        20

          1         20        0         15

          2         20        15        0

          Unidirectional copy peak bandwidth GB/s

          D/D       0           1           2

          0         N/A         14.250      14.251

          1         14.288      661.562     30.770

          2         14.287      30.766      682.556

          Bidirectional copy peak bandwidth GB/s

          D/D       0           1           2

          0         N/A         9.775       9.775

          1         9.775       N/A         57.563

          2         9.775       57.563      N/A

just seeing this, @darren-amd
the BW values seem what's mentionend for cpu-to-gpu transfers in the datasheet/the announcements.
for the Radeon Pro VII it read:
  "communication speed of up to 168 GB/s peer-to-peer between GPUs"

the 64GB/s as peak seem more like the PCIe 4.0 CPU-GPU limit, as per the MI100 datasheet:
"The AMD Instinct MI100 GPU is designed to support the latest PCIe Gen 4.0

technology which provides up to 64GB/s peak theoretical transport data
bandwidth from CPU to GPU per card."
I also wondered if the issue with rocm-smi --shownodesbw was ever fixed?


init: implemented parallel frame processing for windows.ps1

Since the Unix counterpart has parallel executions for video processing, it's time to implement for windows.sp1. This only affects the older implementations so that the project can get going while having the new implementations unaffected.
Joly0 is the sole author of this patch. To awards his GitHub contribution and due to GitHub's strict requirement, this patch is written as "Co-authored" to make sure Joly0 gains the necessary credit for his profile badge.
This patch implements parallel frame processing for windows.ps1 in init/ directory.
Accepted-by: (Holloway) Chew, Kean Ho<EMAIL_ADDRESS>Reworded-by: (Holloway) Chew, Kean Ho<EMAIL_ADDRESS>Reviewed-by: (Holloway) Chew, Kean Ho<EMAIL_ADDRESS>

Code-wise, it's clear. I will merge it in first.


GPR and VGP not agreeing (Gaussian, noiseless, small lengthscale)

I have found that GPR and VGP can produce completely different results in the noiseless case (i.e. the likelihood variance is fixed as a very small number), espescially if the lengthscale of the data is very small (very wiggly).
I'm assuming this is an issue with the optimiser?
Below is a MWE which shows completely different results for a setting I would assume should be easy for VGP to replicate. GPR interpolates the data as desired, VGP does not (not is it close).
import numpy as np #because numpy is always necesarry for everything in python
import matplotlib.pyplot as plt
import gpflow
import tensorflow as tf
import random
random.seed(10)

#lets make some toy function
def toy_sim(x):
    return(np.sin(80*x)*np.cos(50*x))

#and get some training data
x = np.linspace(0,1, num = 100)
x = np.reshape(x, [np.shape(x)[0], 1]) #gpflow wants everything as a matrix
y = toy_sim(x)

#standardise
sdy = np.std(y)
meany = np.mean(y)
y = (y - meany)/ sdy

#now lets fit a standard GP emulator using gpflow

k = gpflow.kernels.SquaredExponential() #specify  kernel:
m = gpflow.models.GPR(data = (x,y), kernel = k, mean_function = None) #create the model:

m.likelihood.variance.assign(0.00001) #make nugget value near zero (can only go so low before numerical issues)
gpflow.set_trainable(m.likelihood, False) #and tell it the nugget is *not* a parameter to be learnt

opt = gpflow.optimizers.Scipy() #create optimsier
opt.minimize(m.training_loss, m.trainable_variables, options = dict(disp = True, maxiter =10000)) #and run optimser

#and then make some predictions
xpred = np.linspace(0,1, num = 1000)
xpred = np.reshape(xpred, [np.shape(xpred)[0], 1]) #gpflow wants everything as a matrix

preds = m.predict_y(xpred)
mean = np.array(preds[0])[:,0] #extract, covnert to numpy array, and convert to vector rather than matrix
var = np.array(preds[1])[:,0]

#plot this:
plt.plot(xpred[:,0], mean)
plt.fill_between(xpred[:,0], mean - 2*np.sqrt(var), mean + 2*np.sqrt(var), alpha = 0.5)
plt.scatter(x[:,0], y, color = "black", zorder = 2) #(zorder makes the points be at the top)
plt.show()
#interpolates the data, as desired

#And then repeat but using VGP

m = gpflow.models.VGP(data = (x,y), kernel = gpflow.kernels.SquaredExponential() , mean_function = None, likelihood = gpflow.likelihoods.Gaussian())

m.likelihood.variance.assign(0.00001) #make nugget value near zero (can only go so low before numerical issues)
gpflow.set_trainable(m.likelihood, False) #and tell it the nugget is *not* a parameter to be learnt

opt = gpflow.optimizers.Scipy()
opt.minimize(m.training_loss, variables=m.trainable_variables, options = dict(disp = True, maxiter =10000))

preds = m.predict_y(xpred)
mean = np.array(preds[0])[:,0] #extract, covnert to numpy array, and convert to vector rather than matrix
var = np.array(preds[1])[:,0]

plt.plot(xpred[:,0], mean)
plt.fill_between(xpred[:,0], mean - 2*np.sqrt(var), mean + 2*np.sqrt(var), alpha = 0.5)
plt.scatter(x[:,0], y, color = "black", zorder = 0)
plt.show()
#completely different - doesn't interpoalte the data at all

I think this is because of the global jitter_level setting.
Numerical stability is a problem in variational methods, since current implementations* need to invert noise-free covariance matrices, which do not have lower-bounded eigenvalues. To avoid Cholesky errors, a small nugget matrix is added. When the observation noise is large, this makes very little difference to the predictions. Sparse methods simply cannot work with zero noise (since they don't have the flexibility to interpolate N > M training points), so usually this all makes sense.
In your case, you're looking at VGP, which is a bit of a special case. Essentially, I believe that mathematically, you should get the same answer if you set the jitter_level to zero in gpflow.config. But the computation will be numerically more unstable, I think.
How come you're comparing GPR to 'VGP? Is there anything you want to use VGPfor that you can't useGPR` for?

Ah, ok so there's two jitters really then? Because I also have the likelihood variance (which serves the same purpose?) as 0.00001. I'm not allowed to make this likelihood variance much smaller before I get a warning (because the softplus transform requires it to be at least 1e-6?)
Setting the gpflow.config jitter to 1e-12 does not help here (I assume because of the other "jitter" which I cannot make smaller?) and setting it to 1e-13 causes numerical issues.
And the reason I posted this issue with VGP, is because I was looking at SVGP with small noise, which has the same issue, and was trying to see at what point the issue pops up (and found it happens even for N==M in SVGP and then even for VGP which surprised me).
I don't expect the sparse approximations to be able to interpolate without all the data, but I was surprised that with N==M (and the inducing points set to the training points) that it performs so abysmally

Hello,
I am profoundly confused: I just run the above code with GPFlow 2.9.2, Tensorflow 2.16.1 and numpy 1.26.4 and the VGP simply will not fit! This is crazy!
My dataset a very large and very "wiggly". It cannot fit to GPU memory without minibatches, thus I am forced to use sparse variational models - but they do not fit despite what jitter values I set (if it does not simply crash due to unstability), not to this example provided by @EvanBaker57 nor to my real life dataset.
So this means I cannot use ANY GP models for my very large very wiggly dataset @markvdw ? I simply refuse to believe that to be the case. Can it be a flaw in GPFlow or some library it relies on? Or can it really be pinpointed to lack on infinite bits representing a floating point number?


How were you able to convert the model from .pth to .mlmodel?

Thank you for the model. I was also trying to convert the NAFNet model from .pth into MLmodel format to us in one of my projects and was wondering how you did the conversion since I am relatively new to this. If you could share me the code or any resources you used for the conversion it would be really helpful for me.
Thank you

Hello! You can give this script a shot:
https://github.com/Vadbeg/NAFNet/blob/main/convert_model_coreml.py
I can't guarantee it will function across all platforms. It performed as expected on a MacBook with an Intel i9 processor but faced compatibility issues on an M1, without any apparent explanation.


Unable to run in Macos Apple silicon

I am not able to run this code in mac Apple silicon. do you have cli option to start the scrapper.
(venv) ➜  educative.io_scraper git:(v3-master) python3 EducativeScraper.py

                Educative Scraper (v3.2.8 Master Branch), developed by Anilabha Datta
                Project Link: https://github.com/anilabhadatta/educative.io_scraper/tree/v3-dev
                Check out ReadMe for more information about this project.
                Use the GUI to start scraping.

DEPRECATION WARNING: The system version of Tk is deprecated and may be removed in a future release. Please don't rely on it. Set TK_SILENCE_DEPRECATION=1 to suppress this warning.
Traceback (most recent call last):
  File "/Users/aaa/workspace/tmp/educative.io_scraper/EducativeScraper.py", line 37, in <module>
    app.run()
  File "/Users/aaa/workspace/tmp/educative.io_scraper/EducativeScraper.py", line 32, in run
    HomeScreen().createHomeScreen(self.version)
  File "/Users/aaa/workspace/tmp/educative.io_scraper/src/UI/HomeScreenGUI.py", line 33, in __init__
    self.app.iconphoto(True, tk.PhotoImage(file=os.path.join(constants.commonFolderPath, "icon.png")))
  File "/Users/aaa/.pyenv/versions/3.9.18/lib/python3.9/tkinter/__init__.py", line 4064, in __init__
    Image.__init__(self, 'photo', name, cnf, master, **kw)
  File "/Users/aaa/.pyenv/versions/3.9.18/lib/python3.9/tkinter/__init__.py", line 4009, in __init__
    self.tk.call(('image', 'create', imgtype, name,) + options)
_tkinter.TclError: couldn't recognize data in image file "/Users/aaa/workspace/tmp/educative.io_scraper/src/Common/icon.png"
(venv) ➜  educative.io_scraper git:(v3-master)

@sushantgupta try installing tkinter from terminal
else comment line 33 in HomeScreenGui.py

@sushantgupta any update on this?

@anilabhadatta I had the same issue and it worked, I just had to run install again before running.

@donadoio i guess the manual steps is better recommended for linux/MacOS env
Seems like some dependencies like tkinter and few more are required to be installed separately.

@donadoio can you test again, latest pull and clean install again.

@anilabhadatta the installation is succesfull and I'm able to run the Educative Scraper however when I click on Login Account I get this error
 2024-03-05 16:11:14,303 - INFO - LoginAccount - LoginAccount initiated...
                            Login your account in the browser...
                            To Terminate, Click on Logout Button

 2024-03-05 16:11:14,303 - INFO - BrowserUtility - Loading Browser...
 2024-03-05 16:11:14,323 - ERROR - LoginAccount - start: 27: BrowserUtility:loadBrowser: 35: Message: Service /Users/isaacdonado/educative.io_scraper/src/ChromeDrivers/mac/chromedriver-mac-arm64/ucchromedriver unexpectedly exited. Status code was: -9

 2024-03-05 16:11:14,324 - ERROR - BrowserUtility - No Browser was open to close via websocket
 2024-03-05 16:11:14,324 - INFO - BrowserUtility - An error occurred while deleting lock files: [Errno 2] No such file or directory: '/Users/isaacdonado/EducativeScraper/UserData42/ucDriver-True'

@donadoio disable the ucdriver settings and try again. Also do git pull first.
ucdriver is not supported in Mac Arm based chips.
Can you confirm if you are using m1 based mac ?

I was using m3 based mac.

@donadoio use latest pull. Selenium base is integrated so shouldn't cause issues


cmd/go: cgo builds don't work with "zig cc"

Hello, I am filing this issue on behalf of an issue reported upstream here: https://github.com/ziglang/zig/issues/7342
@lu4p writes:

I would like to use "zig cc" for easily cross compiling cgo dependent go code, however the "drop-in" c compiler replacement doesn't work.
Example cgo code
main.go
package main

//int Add(int a, int b){
//    return a+b;
//}
import "C"
import "fmt"

func main() {
	a := C.int(10)
	b := C.int(20)
	c := C.Add(a, b)
	fmt.Println(c) // 30
}

other compilers
go build main.go //build works fine (uses gcc)
CC="clang" go build main.go //build works fine (uses clang)

zig cc
$ CC="zig cc" go build main.go
# runtime/cgo
info: Usage: zig [command] [options]

Commands:

  build            Build project from build.zig
  build-exe        Create executable from source or object files
  build-lib        Create library from source or object files
  build-obj        Create object from source or assembly
  cc               Use Zig as a drop-in C compiler
  c++              Use Zig as a drop-in C++ compiler
  env              Print lib path, std path, compiler id and version
  fmt              Parse file and render in canonical zig format
  init-exe         Initialize a `zig build` application in the cwd
  init-lib         Initialize a `zig build` library in the cwd
  libc             Display native libc paths file or validate one
  run              Create executable and run immediately
  translate-c      Convert C code to Zig code
  targets          List available compilation targets
  test             Create and run a test build
  version          Print version number and exit
  zen              Print zen of zig and exit

General Options:

  --help           Print command-specific usage

error: unknown command: -E

go version go1.15.5 linux/amd64

So this looks like go is inserting flags before it passes cc. The user will have to work around this by creating a shell script like this:
#!/bin/sh
zig cc $@

And then passing that as CC. This way zig gets invoked with zig cc -E ... instead of zig -E ...
This issue is a feature request for go to honor the entire command line in CC, appending flags rather than prepending them. Or, at least adding explicit detection for zig cc and passing the cc as the first arg.

This seems closely related to #41400, but with the extra caveat that the first argument is not actually the full C compiler command.

CC (ha!) @ianlancetaylor

Related topic, can anyone help shed some light on why zig cc gets EACCES when it tries to create a project-local zig-cache/ directory?
mkdirat(AT_FDCWD, "zig-cache", 0755 = -1 EACCES (Permission denied)

@andrewrk run go build -x to show the build commands
It should be cding to a temp dir before running cc
When I do CC="gcc gcc2" go build -x (ignore the invalid CC string)
I get ... TERM='dumb' gcc -I "c:\\go\\src\\runtime\\cgo" gcc2 -m64 -mthreads -fmessage-length=0 "-fdebug-prefix-map=$WORK\\b057=/tmp/go-build" -gno-record-gcc-switches -I "$WORK\\b057\\" -g -O2 -Wall -Werror -o "$WORK\\b057\\_x001.o" -c _cgo_export.c
After skimming the CL for #41400 it does look like the fix would be in the same places

@bcmills @ianlancetaylor
If we use get rid of string splitting for CC/CXX completely (return []string{os.Getenv("CC")}) we could prob fix this and #41400 without too much complexity.

@andrewrk run go build -x to show the build commands

Hmm I got this:

[nix-shell:~/misc/gozig]$ CC="zcc" go build -x main.go
WORK=/run/user/1000/go-build735552327
mkdir -p $WORK/b001/
cat >$WORK/b001/importcfg.link << 'EOF' # internal
packagefile command-line-arguments=/home/andy/.cache/go-build/24/2472355c95ff5bd2c8379130112402791828be5fa411080b10c62f6c57556a65-d
packagefile fmt=/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/linux_amd64/fmt.a
packagefile runtime/cgo=/home/andy/.cache/go-build/9b/9b3a75664ee02a7fb873a1dc93cf7958c170dedef5fc1c46e2de82b40856bdec-d
packagefile syscall=/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/linux_amd64/syscall.a
packagefile runtime=/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/linux_amd64/runtime.a
packagefile errors=/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/linux_amd64/errors.a
packagefile internal/fmtsort=/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/linux_amd64/internal/fmtsort.a
packagefile io=/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/linux_amd64/io.a
packagefile math=/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/linux_amd64/math.a
packagefile os=/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/linux_amd64/os.a
packagefile reflect=/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/linux_amd64/reflect.a
packagefile strconv=/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/linux_amd64/strconv.a
packagefile sync=/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/linux_amd64/sync.a
packagefile unicode/utf8=/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/linux_amd64/unicode/utf8.a
packagefile internal/bytealg=/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/linux_amd64/internal/bytealg.a
packagefile internal/oserror=/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/linux_amd64/internal/oserror.a
packagefile internal/race=/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/linux_amd64/internal/race.a
packagefile internal/unsafeheader=/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/linux_amd64/internal/unsafeheader.a
packagefile internal/cpu=/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/linux_amd64/internal/cpu.a
packagefile runtime/internal/atomic=/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/linux_amd64/runtime/internal/atomic.a
packagefile runtime/internal/math=/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/linux_amd64/runtime/internal/math.a
packagefile runtime/internal/sys=/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/linux_amd64/runtime/internal/sys.a
packagefile internal/reflectlite=/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/linux_amd64/internal/reflectlite.a
packagefile sort=/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/linux_amd64/sort.a
packagefile math/bits=/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/linux_amd64/math/bits.a
packagefile internal/poll=/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/linux_amd64/internal/poll.a
packagefile internal/syscall/execenv=/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/linux_amd64/internal/syscall/execenv.a
packagefile internal/syscall/unix=/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/linux_amd64/internal/syscall/unix.a
packagefile internal/testlog=/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/linux_amd64/internal/testlog.a
packagefile sync/atomic=/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/linux_amd64/sync/atomic.a
packagefile time=/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/linux_amd64/time.a
packagefile unicode=/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/linux_amd64/unicode.a
EOF
mkdir -p $WORK/b001/exe/
cd .
/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/tool/linux_amd64/link -o $WORK/b001/exe/a.out -importcfg $WORK/b001/importcfg.link -buildmode=exe -buildid=YCcuUkfsSqMNZpM94Hzz/WNU9rIi2KCHzwXiokjH7/pwW5amiz-sk8p-Zpyuhd/YCcuUkfsSqMNZpM94Hzz -extld=zcc /home/andy/.cache/go-build/24/2472355c95ff5bd2c8379130112402791828be5fa411080b10c62f6c57556a65-d
/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/tool/linux_amd64/buildid -w $WORK/b001/exe/a.out # internal
# command-line-arguments
warning: unsupported linker arg: --compress-debug-sections=zlib-gnu
cp $WORK/b001/exe/a.out main
rm -r $WORK/b001/

I don't see the zig cc build commands

@andrewrk if it's been build cached you'd need to run go build-x -a to force a rebuild

Hmm, I'm not sure whether this is an issue or not but here's what I'm seeing:
...
cd /nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/src/syscall
/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/tool/linux_amd64/asm -p syscall -trimpath "$WORK/b029=>" -I $WORK/b029/ -I /nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/include -D GOOS_linux -D GOARCH_amd64 -o $WORK/b029/asm_linux_amd64.o ./asm_linux_amd64.s
/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/tool/linux_amd64/pack r $WORK/b029/_pkg_.a $WORK/b029/asm_linux_amd64.o # internal
/nix/store/4jal9hawn76qvls4wnhcjk9n56a761vm-go-1.15.5/share/go/pkg/tool/linux_amd64/buildid -w $WORK/b029/_pkg_.a # internal
# runtime/cgo
attempt to unwrap error: ReadOnlyFileSystem
...

User runs command with cwd set to $HOME/my-project-directory
Go changes the cwd to /share/go/src/syscall
Zig does this:

mkdirat(AT_FDCWD, "zig-cache", 0755) = -1 EACCES (Permission denied)

Ultimately, since go does its own caching, it would be best for the user to disable zig's caching for this use case. So I'm not sure it really matters. But anyway that's the next thing to solve for this use case.

Just a passing thought, but assuming that the compiler's working directory is writeable might be flawed? At least, it seems reasonable enough to execute a C compiler from a non-writeable directory (maybe you're compiling sources in someone else's home directory?) as long as the output file you've specified is writeable. Maybe it would generally make sense for zig cc to gracefully handle cases where it can't create a local cache in the working directory in general, and not just for cgo?

I solved the cache directory issue with https://github.com/ziglang/zig/commit/f7d600675c548b4e45b24f8e10ebdd892a6b8dba. So, as soon as https://golang.org/cl/276412 lands, I believe this issue can be considered to be solved.

@andrwrk, as a workaround in the meantime you could probably set CC to a small script that invokes zig cc with the same arguments.

@bcmills that workaround is mentioned in the original issue description. :)

resolved with #43808 and CL 284780

resolved with #43808 and CL 284780

@seankhliao #43808 is a separate issue?

@seankhliao #43808 is a separate issue?

oops

oops

resolved with #43808 and CL 284780

yeah running with CL 284780 and that definitely does NOT solve this issue.

I can confirm this fixes the issue.


Keyvault client fails to get secret after migrating keyvault to another tenant

We recently migrated our subscription and all resources including keyvault to another tenant. We did setup all necessary permissions but keyvault fails to get secret with below exception.
Operation execution on the AcisRuntime failed.
Error while retrieving key vault certificate. Identifier:MetadataServiceClientAdminCertificate. Path:https://dummyPath/secrets/Dummycert. Error: System.AggregateException: One or more errors occurred. ---> System.Exception: Failed to authenticate to AAD using certificate thumbprints: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
at Microsoft.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at Microsoft.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccess(Task task)
at Microsoft.Azure.KeyVault.KeyVaultClient.d__cb.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.WindowsAzure.Wapd.Acis.AcisCommon.Secrets.KeyVaultHelper.d__11.MoveNext() in d:\dbs\sh\easl\0616_191330\cmd\2\src\acis\aciscommon\Secrets\KeyVaultHelper.cs:line 120
--- End of inner exception stack trace ---
at System.Threading.Tasks.Task`1.GetResultCore(Boolean waitCompletionNotification)
at Microsoft.WindowsAzure.Wapd.Acis.AcisCommon.Secrets.KeyVaultHelper.TryGetCertificate(String path, String& base64Cert, String& errorMessage) in d:\dbs\sh\easl\0616_191330\cmd\2\src\acis\aciscommon\Secrets\KeyVaultHelper.cs:line 48
---> (Inner Exception #0) System.Exception: Failed to authenticate to AAD using certificate thumbprints: 2FDAA5F17076A0A28B2A126391513B819A95483B
at Microsoft.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at Microsoft.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccess(Task task)
at Microsoft.Azure.KeyVault.KeyVaultClient.d__cb.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.WindowsAzure.Wapd.Acis.AcisCommon.Secrets.KeyVaultHelper.d__11.MoveNext() in d:\dbs\sh\easl\0616_191330\cmd\2\src\acis\aciscommon\Secrets\KeyVaultHelper.cs:line 120<---
.
Server stack trace:
at System.ServiceModel.Channels.ServiceChannel.ThrowIfFaultUnderstood(Message reply, MessageFault fault, String action, MessageVersion version, FaultConverter faultConverter)
at System.ServiceModel.Channels.ServiceChannel.HandleReply(ProxyOperationRuntime operation, ProxyRpc& rpc)
at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs, TimeSpan timeout)
at System.ServiceModel.Channels.ServiceChannelProxy.InvokeService(IMethodCallMessage methodCall, ProxyOperationRuntime operation)
at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message)
Exception rethrown at [0]:
at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)
at Microsoft.WindowsAzure.Wapd.Acis.SME.Runtime.IAcisRuntimeController.PrepareExtension(String extensionName, String packageIdentifier, String packagePath, String tempCopiedPath, String outputPath, String packageTemp, String endpointName, Boolean isLpuDisabledForExtension, Boolean isSslValidationDisabled, Boolean isPackageUpload, Boolean shouldInstallCert, Boolean shouldValidateExtension, Boolean isSecretLoadScopedToEndpoint)
at Microsoft.WindowsAzure.Wapd.Acis.SME.AcisRuntimeManager.CreateFromStorage(String extensionName, String extensionIdentifier, String endpointName, Boolean createLocalRuntime, String jobId) in d:\dbs\sh\easl\0616_191330\cmd\3\src\acis\acissme\RuntimeManager\AcisRuntimeManager.cs:line 565
at Microsoft.WindowsAzure.Wapd.Acis.SME.AcisRuntimeManager.LoadRuntime(String extensionName, String packageIdentifier, String endpointName, String jobId) in d:\dbs\sh\easl\0616_191330\cmd\3\src\acis\acissme\RuntimeManager\AcisRuntimeManager.cs:line 250
at Microsoft.WindowsAzure.Wapd.Acis.AcisJobManager.GetRuntime(String extensionName, String packageIdentifier, String endpointName, String& runtimeId, String& uniqueJobId) in d:\dbs\sh\easl\0616_191330\cmd\3\src\acis\acissme\RuntimeManager\AcisJobManager.cs:line 242
at Microsoft.WindowsAzure.Wapd.Acis.AcisJobManager.ExecuteOperation(IAcisSMEUser user, String operationId, String extensionName, String packageIdentifier, MetaEndpoint endpoint, String operationFlags, MetaOperation operation, List`1 paramValues, Int32 batchOpIndex, AcisRuntimeStats& stats, Boolean isRhcOperation) in d:\dbs\sh\easl\0616_191330\cmd\3\src\acis\acissme\RuntimeManager\AcisJobManager.cs:line 110
Once we restart all nodes it starts fetching correct authority from path and able to get token.
We tried to force refresh AAD token like below.
private async Task<Secret> GetSecretWithRefreshTokenAsync(string path)
        {
            try
            {
                return await (_keyVaultClient.Value.GetSecretAsync(path));
            }
            catch (Exception ex)
            {
                TokenCache.DefaultShared.Clear();
                _keyVaultClient = new Lazy<KeyVaultClient>(
                () => new KeyVaultClient(_aadAccessTokenHandler.Value.GetAccessTokenAsync));
                return await (_keyVaultClient.Value.GetSecretAsync(path));
            }
        }

still does not work as it looks like KeyvaultClient itself caches the authority extracted from path and I do not see a way to force clear authority cache in KeyvaultClient.

Thanks for this feedback. I have routed your request to the appropriate team for follow-up.
/cc: @schaabs

@smitshah91 Thank you for the detailed report.  This is a  known issue https://github.com/Azure/azure-sdk-for-net/issues/5702, and unfortunately there is no way to force the KeyVaultClient to refresh the Authority.  In fact the authority for each vault is cached staticly so restarting the process is the only way to clear the cache.
This issue should be fixed in the next release of the SDK, but until then restarting the process should fix the failure you're seeing.  Please let me know if this doesn't fix your issue.

This issue has been fixed in This issue has been addressed and this fix has been published in v3.0.4 (https://www.nuget.org/packages/Microsoft.Azure.KeyVault/3.0.4). This fix updates the challenge class whenever it receives a 401 response with the WWW-Authenticate header. The resulting behavior when a vault is migrated from one tenant to another is that the first call to the KeyVaultClient after the migration will still call the authentication callback with the original authority, and so that call will still fail with a 401 and the user would have to handle the exception. However subsequent calls to the KeyVaultClient will pass the correct authority to the authentication callback.


Add Supports to System Message to Language Model

Some models do not support System Message. In Language Model is created a boolean that gives this information.

@papayalabs can you look at this one failing test? I don't see an obvious problem with the code or the test

@krschacht Hey!
I wrote this 2 tests on RegisterableTest:
`  test "it creates some language models with supports_system_message enabled and some without" do
people(:ali_invited).update!(personable_type: "User", personable_attributes: { first_name: "John", last_name: "Doe", })
assert people(:ali_invited).user.language_models.where(supports_system_message: false).present?
assert people(:ali_invited).user.language_models.where(supports_system_message: true).present?
end
test "make sure gpt-best(first language model) has supports_system_message: true" do
people(:ali_invited).update!(personable_type: "User", personable_attributes: { first_name: "John", last_name: "Doe", })
assert_equal people(:ali_invited).user.language_models.first.supports_system_message, true
end`
And supports_system_message is true but when reach the test (edit should have supports_system_message checkbox checked) comes as false.

Hey @krschacht! Now all tests pass.

All looks good, merging now., sorry for being MIA a couple days.


Separating the dialog callback interface

Can't the dialog callback interface be separated from the activity such that I can supply my own implementation of the ColorPickerDialogListener independent of the activity I'm using it in? Maybe something like this:
ColorPickerDialog.newBuilder().setColor(color).show(Activity); can be changed to ColorPickerDialog.newBuilder().setColor(color).show(Activity, ColorPickerDialogListener);

Yes, you can already do this. Example:
ColorPickerDialog dialog = ColorPickerDialog.newBuilder().setColor(Color.RED).create();
dialog.setColorPickerDialogListener(new ColorPickerDialogListener() {
  @Override public void onColorSelected(int dialogId, @ColorInt int color) {
    // do something
  }

  @Override public void onDialogDismissed(int dialogId) {
    // do something
  }
});
dialog.show(getFragmentManager(), "ColorPickerDialog");

If you are using ColorPreference then you shouldn't have to worry about the callback.

Thanks :D and sorry for not checking it out thoroughly. I think you should document this code in the README too :)


Added max_files parameter to extract_features_and_train

Currently, extract_features_and_train needs a list of folder paths. It would be useful to be able to set how many files per folder to read at most. So I added max_files parameter with default 1000. Potentially randomly choosing those files would be another addition.
I tested it in a Kaggle notebook and it worked fine.
Motivation behind it was that there is a Birdcall Kaggle competition with 264 classes (folders) and ~100 files per class (folder). It took longer longer than 9 hours to train a model and the Kaggle notebook timed out. So I decided to train on smaller number of files per folder, i.e. undersample classes.
from pyAudioAnalysis import audioTrainTest as aT

# train classifier
train_folder = '../input/birdsong-recognition/train_audio/'
mid_term_window_length = 2
mid_term_window_step = 1

# get audio file folders
class_paths = [train_folder + x for x in sorted(os.listdir(train_folder))]

model_name = "../input/bird-call-classification/svmSMtemp"
model_type = 'svm'

if not os.path.exists(model_name):
    model_name = "svmSMtemp"
    # train classifier using folders of audio files
    aT.extract_features_and_train(class_paths,
                                  mid_term_window_length,
                                  mid_term_window_step,
                                  aT.shortTermWindow,
                                  aT.shortTermStep,
                                  model_type,
                                  model_name,
                                  False,
                                  max_files = 5)

Analyzing file 1 of 5: ../input/birdsong-recognition/train_audio/aldfly/XC134874.mp3
Analyzing file 2 of 5: ../input/birdsong-recognition/train_audio/aldfly/XC135454.mp3
Analyzing file 3 of 5: ../input/birdsong-recognition/train_audio/aldfly/XC135455.mp3
Analyzing file 4 of 5: ../input/birdsong-recognition/train_audio/aldfly/XC135456.mp3
Analyzing file 5 of 5: ../input/birdsong-recognition/train_audio/aldfly/XC135457.mp3
Feature extraction complexity ratio: 28.2 x realtime
Analyzing file 1 of 5: ../input/birdsong-recognition/train_audio/ameavo/XC133080.mp3
Analyzing file 2 of 5: ../input/birdsong-recognition/train_audio/ameavo/XC139829.mp3
Analyzing file 3 of 5: ../input/birdsong-recognition/train_audio/ameavo/XC139921.mp3
Analyzing file 4 of 5: ../input/birdsong-recognition/train_audio/ameavo/XC155039.mp3
Analyzing file 5 of 5: ../input/birdsong-recognition/train_audio/ameavo/XC166076.mp3
Feature extraction complexity ratio: 27.5 x realtime

Thanx for the PR @KobaKhit
It would be nice if
(a) default value was -1 which indicates that no max files is used in the feature extraction process
(b) random shuffling would also be parametrized (not by default set to true, as in many cases we need the feature extraction to take place in the file path order)


Add a changelog

Pick a format and start maintaing a changelog
Maybe from https://keepachangelog.com/en/1.0.0/
I don't think it formats nicely on GitHub though so maybe I'll look around. Bonus points if I can just automate it via a gh action or something.

Added a release.yml that will generate release notes. Maybe that's enough?


Frontend/fix/map height too big

Should fix #1048, but I can't test on mobile outside of the dev tools
With extra padding, the map is cut off at the bottom, and you need to scroll to see it all (even though its 100vh - navPaddingMobile)

Without it sits nicely

All of the pages had this weird extra top padding

Which they don't now

And the only one benefitting from it was the profile page

Which would have been cut off with this fix

But i've re-added the padding just to this one, so it looks the same as it did before

Hi, I'm new here so forgive me if I did something wrong but I've tested this on android/chrome using the preview url
https://085e6c07--frontend.preview.couchershq.org/map#loc=1/52.2/9.4
and saw that the map was still cut off:

Probably because google chrome doesn't take the height of the address bar into account when using height: 100vh. Note that you can scroll down (by dragging on the small area on eather side of the map)  to hide the address bar, then the map nicely fits the entire screen.
You can set a height of 100% to each and every parent (including the body and html tag) to make sure the map will be visible. However then you won't be able to scroll down to hide the address bar anymore.
Another idea might be to scroll down when using one finger and pan the map when using two fingers. (although I usually don't like this behavior)

Thanks for taking a look @thdk! Yeah unfortunately [100vh seems to be quite well known to not work well on mobile](https://chanind.github.io/javascript/2019/09/28/avoid-100vh-on-mobile-web.html) so I think what you suggested might be the best solution without resorting to a JavaScript based one which I don't really like either for something that feels like should be solvable by CSS these days...

Ha thanks @thdk! So it seems all my fix did was remove the extra padding.
@darrenvong I wasn't sure whether it was a deliberate design choice to have that much padding, shall I revert it?
Also, do we really want the map to cover the whole screen anyway? Why not set the map height to 90vh instead? Then it'll fit regardless of whether the url bar is visible

Why not set the map height to 90vh instead

I like this. Why use lots of code when little code does trick? (It's not robust, but I think a robust version solves a problem that doesn't exist)

I think the map doesn't need fill the entire height as well. So we could indeed reduce the map height (for all browsers?) or we can leave it like it is because that's how google decided it should work :)

Actually we may need to have it accurately 100vh when we have the proper map design, for the fullscreen effect...
In this case I think it can be achieved with some useLayoutEffect and innerHeight. Not ideal, but it will be difficult to set 100% in html and body for only one route (unless we can just do it for all routes? Does it break scrolling?)

it will be difficult to set 100% in html and body for only one route
Why only one route? I don't think setting that at least at the html and body is a bad idea since height doesn't "inherit" like width does, and it makes building certain layouts easier if we want to keep height proportional, even though it might means we have to "drill" height down, but hopefully not too many levels for intentional full screen layout?

After reading about the "100%" on body and html thing, I think it's probably preferable. Do you want to try and see which is nicer?

h1 has 1.5rem margin top

I have to say this is kinda annoying me, as in some places (lol aka the discussion stuff I'm working on) I had to override it again since it creates undesirable spacing at the top so I agree with removing it from the h1.
This article might have some strong opinions about margins as I'm a bit iffy about adding junk elements for spacing, but sums up my thoughts when it comes to reusable components having its own margin. The whole margin collapse thing is an annoyance that adds to your mental overhead to think about when you try to use a component with margin and lay things out with other things, when I'd rather plug it in a component/page specific container and let that deal with spacing concerns.

True. Maybe then let's remove all those marginTops from the theme on the headers.

Oke doke my dudes. I'll switch it all up 👌

cool @lucaslcode @darrenvong I've done that
After removing the marginTop from the theme headers, I saw that actually the 1.5rem was coming from PageTitle. I've changed it from margin to padding which helps with the mental overhead, but can remove it all together if you want

Hmm ok we're not there just yet. I have the map still overflowing on mobile,
And then when set height: 100% the rest of the way down from body to the map, it thinks height is 0

Cool, that's working now

Looks good to me too! Quite happy with how simple this turns out once all the JS are removed :)

Yeah me too, so much cleaner!


Is it possible or have a plan to support general SQL Explorer?

It seems only support SQL Explore for Django app/scheme. Is there any plan to support general SQL explore just like phpmyadmin?

Absolutely! In fact -- there is a very functional branch for doing exactly that. See:
https://github.com/groveco/django-sql-explorer/pull/247
It's pretty much ready for prime time except for this one weird failing test. I will get around to releasing it soon.


fix(getTokenData): JSON controller accepts flag for showing tx history

:tada: This PR is included in version 2.7.1 :tada:
Your semantic-release bot :package::rocket:


⚠ Bofrost.lu has degraded performance

In b9a3a51, Bofrost.lu (https://www.bofrost.lu) experienced degraded performance:

HTTP code: 200
Response time: 1157 ms

Resolved: Bofrost.lu performance has improved in edd1569 after 16 minutes.


[Questions] - VerifyTotp is incorrect?

Tks for your libs so much

I'm using "ComputeTotp" method to generate Otp token with step 30s (expired otp). However I'm check "VerifyTotp" method to verify gererated otp token is valid or not but after 30 seconds the generated otp token is still valid and nearly 1 minute expired after that.
So I'd like to ask you is it correct ? and how to config expired time exactly.

Tks and have a nice day

Is your system time correct on both devices?

Sorry for misunderstanding.
I'll close it right now.
Tks so much


Minor Update for typo

Pull Request Test Coverage Report for Build 699

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 52.114%

Totals

Change from base Build 695:
0.0%

Covered Lines:
604

Relevant Lines:
1159

💛  - Coveralls

Thanks for the eyes on fixing this one up! Appreciate it big time, ship it!


decreaseIndentPattern doesn't always take effect

While writing a language extension (not a server) I ran into an apparent issue with how decreaseIndentPattern in IndentationRules is applied: it only outdents if you press enter before the matched characters, not after (say, at the end of that line). I was expecting it to act on the entire line's text regardless of where on the line the cursor was.
Using version 0.10.6, commit a80232bbcfe8a5cdad1ebc98638673d9dcb02458. There are no errors in the developer console and I have no extensions (besides the test extension and what VS Code comes with) installed or running.
Reproduction steps:

Create a basic TypeScript extension. I made mine using the Yeoman generator.
Set it up so it adds a new language and activates when that language is selected.
Set that language's configuration on activation, using these indentation rules:

indentationRules: {
    decreaseIndentPattern: /^\s*outdent\s*$/,
    increaseIndentPattern: /^\s*indent\s*/,
}

Run the extension and select the language.
Type indent into the file (or otherwise indent)
Type outdent on a new line and press enter with the text cursor at the end of the text. Observe that it does not outdent.
Type outdent on another indented line and press enter with the cursor before the text. Observe that it does outdent as expected.

I threw together an example extension that shows this issue. Here's a GIF from that:

I need to read again https://manual.macromates.com/en/appendix and confirm

+1. have this bug also (on 0.10.6)
Any chances to include it in JAN-2016 release?

I'm sorry, we're wrapping up January today and I still have other critical issues to look at.

I've looked again at https://manual.macromates.com/en/appendix and I think there is indeed something missing on our side - specifically our assumption that deciding what to do is localized to the current line being edited.
Considering the following rules (from their example):
increaseIndentPattern = "^.*\{[^}\"']*$";           // intends to target lines ending in {
decreaseIndentPattern = '^(.*\*/)?\s*\}[;\s]*$';    // intends to target lines ending in }

Here are some examples I can think of (| indicates cursor when Enter is pressed):
// A) ------------------------------
// BEFORE
int main (int argc, char const* argv[])|{
// AFTER (do nothing)
int main (int argc, char const* argv[])
|{

// B) ------------------------------
// BEFORE
int main (int argc, char const* argv[]) {|
// AFTER (indent new line)
int main (int argc, char const* argv[]) {
    |

// C) ------------------------------
// BEFORE
    if (true)|{}
// AFTER (do nothing)
    if (true)
    |{}

// D) ------------------------------
// BEFORE
    if (true) {}|
// AFTER (do nothing)
    if (true) {}
    |

// E) ------------------------------
// BEFORE
    if (true) {|x-=5;}
// AFTER (indent new line)
    if (true) {
        |x-=5;}

// F) ------------------------------
// BEFORE
        x-=5;|}
// AFTER (outdent new line)
        x-=5;
    |}

// G) ------------------------------
// BEFORE
        x-=5;
    }|
// AFTER (do nothing)
        x-=5;
    }
    |

// H) ------------------------------
// BEFORE
        if (true) {
        }|
// AFTER (do nothing)
        if (true) {
        }
        |

// I) ------------------------------
// BEFORE
        x-=5;
        }|
// AFTER (outdent current line + new line)
        x-=5;
    }
    |

// J) ------------------------------
// BEFORE
        if (true) {
            }|
// AFTER (outdent current line + new line)
        if (true) {
        }
        |

As far as I can tell, all would work today in VS Code except I) and J). Of interest the difference between H) and J) where the lines would match the same regexes, but differ in terms of indentation, thus producing different results.

I ran into the same issue while writing an indent-enabled extension for the Julia language. It's quite common to write:
function foo()
end

with a manual [enter] after foo(), followed by end[enter]. But because of the way this issue works out, I instead end up with:
function foo()
    end

With the current API it seems somewhat difficult to get the intended behavior, while, for example, Atom automatically unindents the end with the same RegExp (even before pressing [enter]).

https://github.com/Microsoft/vscode/pull/16568 aims to mitigate this issue.

@Allardvm you can now set editor.autoIndent to true in today's Insider to embrace auto indent on type. The end will decrease if you put that in your decreaseIndentPattern rule.


I see the chat model file. But where is the stable diffusion image?

What is this local files only flag? And AutoPipelineForText2Image? It’s like it gets the image from somewhere magically. But hey, it is running. Chat and image.

The local_files_only=True flag is for don't try to download model if they already downloaded (in the Hugging Face cache) and the AutoPipelineForText2Image is for load the Stable Diffusion model into the RAM/VRAM.


Docker

@carlsondev @david-shearon Should the workflows be updated to use this docker container now? Is that possible to do without publishing to the Docker container hub thing?

Yes, it is possible. Maybe look here for reference: https://github.com/JonasVautherin/px4-gazebo-headless/tree/master/.github/workflows

I was referring to the integration tests workflow, which requires having a MAV SDK server instance running alongside it, not the building and deploying of the docker container. This requires starting up the docker container as a service and establishing a network. It shouldn't be that difficult, but in practice, I haven't had any luck.
@justincdavis All the other containers don't need it because they don't need all the extra PX4 stuff and they just need a base ubuntu.

@david-shearon Can we actually update this container to use the .devcontainer framework that Github supports?
@carlsondev This could fix some of your issues as well.
https://docs.github.com/en/codespaces/setting-up-your-project-for-codespaces/introduction-to-dev-containers

@justincdavis I'll look into it. Does that just entail changing the name of the directory?

@david-shearon That and adding a docker.json or similar file (see the link). No changes to the Dockerfile are required

@david-shearon @carlsondev I think this blogpost thing could be quite useful :)
https://bradyjoslin.com/blog/dev-action-container

@justincdavis, @carlsondev  helped me with the dev container. Is this what you were looking for? It doesn't build the docker container for some reason.


[5.5] Use dropAllTables

use drop all tables instead of rollback in database migrations trait

This changes the behaviour

@ntzm why?

It's dropping tables not doing rollback. It's not the same.

@decadence ah yeah. but do we really need to use rollback for testing since we always want to clean the database end of the test. rollback uses multiple queries to finish the task this improves the speed of testing. is this unnecessary feature or any suggestions to improve this?

Because rolling back doesn't necessarily mean dropping all the tables. All it does it run down on each migration

@ntzm what about using a different trait like FreshDatabase?

People can want to test their down methods instead of just clearing database.

There's no need for any of this - there is a new trait in 5.5 called RefreshDatabase that already does all of this:
https://github.com/laravel/framework/blob/master/src/Illuminate/Foundation/Testing/RefreshDatabase.php

Didn't know about that @laurencei, thanks


chore: remove logging of homefeed response

Description
While debugging using someone's logs the other day, I found myself wading through tons of unnecessary lines of blockchain-api transaction responses. The response was being logged after every data fetch (every 10 seconds), which I think is unnecessary and bloats the stored file size.
Test plan
n/a
Related issues
n/a
Backwards compatibility
Y
Network scalability
If a new NetworkId and/or Network are added in the future, the changes in this PR will:

[ ] Continue to work without code changes, OR trigger a compilation error (guaranteeing we find it when a new network is added)

🙌 This reminds me, we could also minimize the logging of the tokens info. It’s also a huge chunk visible on each app refresh during dev.

oh yes, our redux logger needs some love...i'll take a pass through that separately


withJson shoult not set http status code implicitly

I thought that withJson set http status code confusing user.
e.g.
$response->withStatus(500)->withJson([])
and
$response->withJson([])->withStatus(500)
are not equally.:cry:
We should fix withJson to intuitive.
Thank you for your consideration.

Unfortunately this would be a backwards incompatible change (it would break the code of anyone using this method when they ran a composer update).

Thank you for the comment.
I thought that default status code to null is better if we need backwards compatibility.

This is still a BC break. Also why wouldn't you do $response->withJson([], 500) if you need a different status code?

@silentworks I can see the issue.
$response = $response->withStatusCode(404)->withJson([]);

This code should return a resposne with a 404 status code.

Please also note that withJson is a helper method, it should help you write less code for a specific purpose.
@AndrewCarterUK that should return a response with 200, if the developer wants it to return a status of 404 either do:
$response = $response->withStatusCode(404)
    ->withHeader('Content-Type', 'application/json;charset=utf-8')
    ->getBody()->write(json_encode([]));

or
$response->withJson([], 404);

It 'does' return that status code, I wouldn't say that it should.
withJson should just give you your response 'with json'. Changing the status code is a non-obvious side effect that I can see causing problems (it already has enough to trigger this PR).

:-1:
You are loading a message into the response and it should return 200 by default.
If you want to change it, it would only make sense if all methods that mimic this functionality were also changed.
The supplying 200 by default is a shortcut for our users.

Can we have a unit test for this please?

Thanks.

{
            "name": "slim/slim",
            "version": "3.3.0",
            "source": {
                "type": "git",
                "url": "https://github.com/slimphp/Slim.git",
                "reference": "939f2e85d57508de9cff241d10091cd972f221c3"
            },
            "dist": {
                "type": "zip",
                "url": "https://api.github.com/repos/slimphp/Slim/zipball/939f2e85d57508de9cff241d10091cd972f221c3",
                "reference": "939f2e85d57508de9cff241d10091cd972f221c3",
                "shasum": ""
            },
            "require": {
                "container-interop/container-interop": "^1.1",
                "nikic/fast-route": "^0.6",
                "php": ">=5.5.0",
                "pimple/pimple": "^3.0",
                "psr/http-message": "^1.0"
            },
            "require-dev": {
                "phpunit/phpunit": "^4.0",
                "squizlabs/php_codesniffer": "^2.5"
            },
            "type": "library",
            "autoload": {
                "psr-4": {
                    "Slim\\": "Slim"
                }
            },
            "notification-url": "https://packagist.org/downloads/",
            "license": [
                "MIT"
            ],
            "authors": [
                {
                    "name": "Rob Allen",
                    "email"<EMAIL_ADDRESS>                    "homepage": "http://akrabat.com"
                },
                {
                    "name": "Josh Lockhart",
                    "email"<EMAIL_ADDRESS>                    "homepage": "https://joshlockhart.com"
                },
                {
                    "name": "Gabriel Manricks",
                    "email"<EMAIL_ADDRESS>                    "homepage": "http://gabrielmanricks.com"
                },
                {
                    "name": "Andrew Smith",
                    "email"<EMAIL_ADDRESS>                    "homepage": "http://silentworks.co.uk"
                }
            ],
            "description": "Slim is a PHP micro framework that helps you quickly write simple yet powerful web applications and APIs",
            "homepage": "http://slimframework.com",
            "keywords": [
                "api",
                "framework",
                "micro",
                "router"
            ],
            "time": "2016-03-10 21:37:40"
        }