Scoping is rather peculiar...

In a shell, define a silly function, f/1, that takes an x and returns the free variable, y.
> (defun f (x) y)
f

Bind local variables x and y to 'undefined and 42, respectively.
> (let ((x 'undefined)
        (y 42))
    (f x))

The result of the previous form is unexpected...
42

When I try to compile a module (say example.lfe) with the f/1 above, I get an error and it preventing me from experiencing this peculiar scoping.
> (c "example.lfe")
./example.lfe:4: unbound symbol: y
error

Perhaps we should check for free variables and prevent functions referring to them from being defined in a shell.

Perhaps. I tend to view it that the functions are all defined inside an environment containing function and macro definitions, and variable values. Like they were all defined in a dynamic fletrec. It needs to be dynamic otherwise the shell would be too static to be useful.
Currently all checking is done at run-time and the evaluator in lfe_eval doesn't trim away variables when calling a function. It is a bit inconsistent with functions in modules but rather practical.

Cool. I support that and also the fact the such functions don't compile when written in files is good enough for me. Cheers.

Sort of off-topic but in Common Lisp the first let statement raises an error even in the shell, but you could (declare (special y)) either in the function or in the let body and then it outputs 42 just as here. I don't know what LFE or Erlang will allow for what Common Lisp calls "dynamic extent/lexical scope" but I'm coming to appreciate it as one of CL's most profound features. I wouldn't feel comfortable suggesting that LFE do exactly what CL does here (even if possible), but whatever LFE can do about regulating and documenting the dynamic and lexical effects would help LFE programmers immensely in the long haul.

The problem in LFE that having a (declare (special y)) form would only work in the REPL. In compiled LFE code there is no provision for dynamically scoped variables and there is no support for it in Erlang. It would be a significant modification to LFE to add it and it would make interfacing Erlang much more complex.
One thing we could do however is to support defparameter and defvar at the top module level. I know of a cunning sneaky way to do this. However it is actually adding global variables to LFE which Erlang doesn't really support.
Erlang doesn't do global or sharing.


The ResizeObserver omits unobserving in the componentWillUnmount, which may lead to memory leak when the component is unmount

The unmounted component can't be GC, because the observer still observes the detached component.

+1


“GoogleMobileAds.xcframework” is not signed with the expected identity and may have been compromised

Since I have updated everything : android, flutter, brew, pod... I have this issue now when running on IOS. It is well working on Android. I have cleaned and relaunched everything.
google_mobile_ads: ^5.2.0
implementation 'com.google.android.gms:play-services-ads:23.4.0'

Hi @carpentierchloe, since flutter plugin version 5.2.0 uses iOS Google Mobile Ads SDK version 11.10.0 which updated the signing identifity in 11.4.0
Per the iOS release notes: Updated the signing identity from AdMob Inc. to Google LLC using team ID EQHXZ8M8AV. If this triggers a notice in Xcode, select “Accept Change” to accept the new identity.


Assign ID3D12Object names during replay to assist debugging

The problem
ID3D12 objects only have names captured if the app called SetName() on them. If the app did not assign names, then debugging replay becomes harder since GFXR does not have any names recorded for them.
The solution
Add a new switch  --dx12-override-object-names for the replayer. Default is off. This will:

Generate a "gfxr_obj_captureId (creator)" string when an object is created during replay
i.e. "gfxr_obj_8 (ID3D12Device8_CreateCommittedResource2)"
Call SetName() using that string on all objects
Bypass any object naming performed by the app

Testing
Verified that object names are generated and set during replay.

CI gfxreconstruct build # 2515 passed.

CI gfxreconstruct build # 2517 running.

CI gfxreconstruct build queued with queue ID 39021.

CI gfxreconstruct build # 2518 running.

CI gfxreconstruct build queued with queue ID 51699.

CI gfxreconstruct build # 2608 running.

CI gfxreconstruct build # 2608 passed.

CI gfxreconstruct build queued with queue ID 53176.

CI gfxreconstruct build # 2626 running.

CI gfxreconstruct build # 2626 passed.

CI gfxreconstruct build queued with queue ID 59215.

CI gfxreconstruct build # 2662 running.


Replace Flake8 with Ruff.

While still in alpha/beta, flake8 is slow and causes overheads during commit. Using a faster tool has great value.

isort can also be removed. PyUpdate also becomes unnecessary.

Done.


add status watcher for bayeux client

add client to watch over status of subscriptions.

:green_heart: Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Start Time: 2022-04-19T13:57:41.129+0000

Duration: 2 min 23 sec

:robot: GitHub comments
To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.


ClusterChecker NPE and Health

Small adjustements following changes to Health-related components that came with Boot 2.2

Change of format and authorization to /health exposed a possible NPE in ClusterCheckerTask
Restore unauthorized access to /health
Fix HealthCheckMetricsAspect
Add new metrics tracking active gRPC streams (FileTransfer and Heartbeat services)

Coverage decreased (-0.03%) to 92.673% when pulling 5b4a51908fede09123843a895007d2e08f533d93 on mprimi:GENIE-531-cluster-checker-npe into eb38dbd8fe50dd2dacf091e29671837040fa2be7 on Netflix:master.


Make .bib file parsing more robust

The bibfish regex parsing did not parse my weird .bib files (but there is a warning for that, so not a surprise). I tried to see if I could expand the regex parsing for my weird .bib files, but regex parsing is rarely robust, and I found BibtexParser which seems pretty functional.
Would you be interested in a PR that adds BibtexParser as a requirement for bibfish (which would allow for more robust .bib file parsing) and uses it to parse .bib files for bibfish?

This is an interesting idea and I'm open to exploring it some more, but I suspect it won't fit so well with my minimalist philosophy. In particular:

I rather like the fact that Bibfish doesn't do any serious parsing of the bibtex/latex because this gives the user maximum flexibility to construct entries however they like. (For example, I just tried running BibtexParser on my database and it didn't like my @presentation and @magazine entries, and it reorders the fields alphabetically).
This would add two dependencies, BibtexParser and PyParsing, which I will have to stay up-to-date with going forward.
BibtexParser seems to be very fast and solid, but it does take a little bit of time to do a full parse of the bibtex – possibly this won't scale well with much larger databases than my own.
BibtexParser has lots of customization options, so we'd have to either (a) enforce some output style or (b) allow the Bibfish user to pass lots of options through to BibtexParser.

I'm curious through: what do your weird bibtex entries look like? Perhaps they can still be accommodated within the simple regex approach.

Firstly, I appear to have citekeys with a period (".") in them, which is a regex special character, which can cause some oddness in the match in extract_bibtex_entries() but that's easily fixed with running re.escape() on the cite key before combining it with the other r-strings and passing the result to re.search().
The more important difference is that not all of my .bib entries end with r"\n}\n".  Sometimes the ending curly brace is on the same line as the last tag in the entry.  There's also nothing preventing a right-curly brace being on a line on its own within a BibTex entry as part of the terminating bracket of a tag as part of the entry, but not the terminating bracket of the entry.
I appreciate that you may not want to take on dependencies, and that's fine.  Designing software is always about picking a path between requirements.  For me, I don't care how the tags in a BibTex entry are arranged in the bibfish output, what's valuable to me is the ability to pick out the entries without having to do it by hand.  If preserving the plain-text arrangement of the tags (and possibly comments) inside a bibtex entry is important to your vision of bibfish and you don't want to take on the BibtexParser dependency, then that's that, and I absolutely respect it.
I'll note that the default parsing strategy from BibtexParser is conservative by design, a parser instance can be instantiated and modifed it to be more ... omnivorous ... so that it would happily consume your @presentation, @magazine, or any other type that it considers non-standard, and for the bibfish use case, I think we'd tune BibtexParser to be as accepting as possible (and not deal with exposing complexity to bibfish users).
I've offered the PR because I need changes for my use case, but I will not be offended at all if you don't want to take on that dependency, which is why I asked, and will be happy to use my own modified version of bibfish.

that's easily fixed with running re.escape() on the cite key

Ah, yes, this we should definitely fix.

The more important difference is that not all of my .bib entries end with r"\n}\n".

Yep, I knew this was a horrible hack when I wrote it. And I'm sure this has presented a problem for many potential users who just gave up.
You're right that, in order to support a wider range of files, Bibfish really needs to do a proper, robust parse of the bibtex. And I can also see that parsing the bibtex would probably have some other side-benefits too – it would simplify some of the code and permit more expansion options in the future. And, if we can make it omnivorous, as you say, then I'd be happy with that.
So, on balance, yes, let's do this.

Okay, cool.  I will craft a rather meaty PR based on these discussions that will add the BibtexParser dependency as well as the multi-bib capability we're talking about in #4.  Might take me a week or so.

Thanks for all your work on this! I have done a little bit of tidying up of PR #6 and pushed a new release (v.0.2.0). Let me know in case there are any issues. The only substantial thing I changed was to remove the messages about the number of citekeys found etc., since I prefer silent functioning by default (but happy to add a verbose option if you like this).

That's fine, I should have cleaned up all of those debugging print statements.


Provide cleaner way to clear the habitat loaded module cache

Hello!
It seems Lemur caches module scripts loaded via the habitat implementation of the "require" method.  This is understandable as it emulates the expected Lua behavior.
However, for unit test purposes it would be nice to have an easy way to clear this that is built right into the library.
Rationale
When unit testing, it's often useful to start certain test cases from a clean slate which is clunky given Lua's (and therefore Lemur's) caching of module scripts.
Example
Consider the following Rodux dispatcher
local FishBagDispatcher = {}
FishBagDispatcher.prototype = {}

local dependencies = require(script.Dependencies).Get()

function FishBagDispatcher.Dispatch(totalFishCaught, bagContents)
    dependencies.VM:dispatch({
        type = "fishBagContentsChanged",
        FishBag = {
            Total = totalFishCaught,
            Contents = bagContents
        }
    })
end

dependencies.FishBagContentsChangedRE.OnClientEvent:Connect(function(newTotalFishCaught, newFishBagContents)
    FishBagDispatcher.Dispatch(newTotalFishCaught, newFishBagContents)
end)

local totalFishCaught = dependencies.GetTotalFishCaughtRF:InvokeServer()
local fishBagContents = dependencies.GetFishBagContentsRF:InvokeServer()

FishBagDispatcher.Dispatch(totalFishCaught, fishBagContents)

return FishBagDispatcher.prototype

I tend to like my unit tests as small, and focused as possible.  Therefore, I would like to show that the total fish caught gets dispatched during loading in one test case, and the contents of the fish bag in another.  Because the module script is cached after the first test case, I need to use a hack to clear the cache for the script.  Though it's a one liner (see setup function below), it's not very expressive, and accessing the metatable just feels gross.
        local function setup()
            getmetatable(script.Parent).instance.moduleLoaded = false

            require(script.Parent.Dependencies).Inject(dependencies)

            dependencies.GetTotalFishCaughtRF.OnServerInvoke = function()
                return totalFishOnServer
            end

            dependencies.GetFishBagContentsRF.OnServerInvoke = function()
                return fishBagContentsOnServer
            end

            dependencies.VM.dispatchedObjectCapture = nil
        end

        it("Should retrieve the total fish caught from the server when loading.", function()
            totalFishOnServer = 11
            fishBagContentsOnServer = nil

            setup()

            require(script.Parent)

            expect(dependencies.VM.dispatchedObjectCapture.FishBag.Total).to.equal(11)
        end)

        it("Should retrieve the contents of the fish bag from the server when loading.", function()
            totalFishOnServer = nil
            fishBagContentsOnServer = {
                StarFish = 2,
                SunFish = 4
            }

            setup()

            require(script.Parent)

            local dispatchedObj = dependencies.VM.dispatchedObjectCapture.FishBag.Contents

            expect(dispatchedObj).to.be.ok()
            expect(dispatchedObj.StarFish).to.equal(2)
            expect(dispatchedObj.SunFish).to.equal(4)
        end)

Would you be open to a nicer way to solve this?  A few ideas are...

Create a "unrequire" method in the habitat which clears the module loaded flag for the provided instance.
Create an optional second parameter to prevent caching

I have this module in my testing suite:
local function RequireWithoutCache(object)
	local result = require(object)
	getmetatable(object).instance.moduleLoaded = false
	return result
end

return RequireWithoutCache

Thanks Kampfkarren.  The solution that both you, and I came up with requires examination of source code internals, and is a hack that exploits lax encapsulation.  Since starting from a clean state is a pretty essential feature for unit testing, it seems like direct, expressive library support would be nice.
Now that I think of it though, do you think this is more an issue that should have been created on the TestEZ repository, since the use case would only be relevant to unit testing, and not necessarily the concern of Lemur?

TestEZ is meant to be run in real Roblox, so I'm not sure how that'd work.
In real Roblox you can cheat the cache by using require(module:Clone()), but Lemur doesn't support Clone yet.
I'm not exactly too sure, but I'd think this is a Lemur issue.

I'm mostly against the idea of extending Lemur or TestEZ to let it support doing things that aren't supported in Roblox. The goal of both projects first and foremost is to allow a faithful testing environment for continuous integration for Roblox, and any divergence hurts that goal!
I think what Kampfkarren suggested -- cloning your module -- is probably the best way to test things that absolutely need module-level side effects or state. In general though, I'd advise not having side effects or mutable module state, because they make testing more difficult and sort of obscure what require does at any given point.

I see where you're coming from.  Require is essentially a method call and my uncaching the call is not a hack because of a missing feature in the library as I originally stated,  but rather a hack to work around a design / implementation flaw (i.e. the module script was written naively,  and resulted in the call to require not being idempotent).
If that's what you meant, I understand and will update my middle scripts.
Did not really think of require as a method call that should be held to the same functional programming standards as any other method.
Thanks for the advice and sorry about another issue that's anything but.

Don't worry about filing issues that don't turn into real issues! It's always great to get feedback on the tools and libraries I work on, regardless of the outcome. Even misunderstandings can point to bad documentation or a slightly off-kilter design. 😄


Update README.md

@KafkaListener annotation changed to @RabbitListener in md-plugin-description

Thanks!


EKS k8s with NLB has TargetGroup unhealty

Summary
Kong deployed via kubectl apply -f https://bit.ly/k4k8s doesn't work.
Kong Ingress controller version
0.8.0
Kubernetes version
Client Version: version.Info{Major:"1", Minor:"17", GitVersion:"v1.17.4", GitCommit:"8d8aa39598534325ad77120c120a22b3a990b5ea", GitTreeState:"clean", BuildDate:"2020-03-12T23:41:24Z", GoVersion:"go1.14", Compiler:"gc", Platform:"darwin/amd64"} Server Version: version.Info{Major:"1", Minor:"15+", GitVersion:"v1.15.11-eks-af3caf", GitCommit:"af3caf6136cd355f467083651cc1010a499f59b1", GitTreeState:"clean", BuildDate:"2020-03-27T21:51:36Z", GoVersion:"go1.12.17", Compiler:"gc", Platform:"linux/amd64"}
Environment

AWS:
Amazon's EKS k8s 1.15 AMI
kubectl apply -f https://bit.ly/k4k8s

Problem
I'm trying do deploy Kong via Kustomize or static yaml. It seems that they both use the AWS Network Load Balancer. For some reason, the target groups are in an unhealthy state. I have no problem running Kong installed via Helm. The difference is that Helm chat is using a classic load balancer there for all works.
Kong controller created by kubectl apply -f https://bit.ly/k4k8s doesn't work.
Kong installed via : helm install kong/kong --generate-name --set ingressController.installCRDs=false  works.
Steps To Reproduce

On Amazon EKS cluster just launch kubectl apply -f https://bit.ly/k4k8s

We have run into the same issue :-(

What healthcheck was configured, and what error condition does it indicate? https://docs.aws.amazon.com/elasticloadbalancing/latest/network/target-group-health-checks.html#check-target-health
Are the cluster subnets properly tagged for load balancer use? https://docs.aws.amazon.com/eks/latest/userguide/load-balancing.html#subnet-tagging-for-load-balancers indicates that this is done automatically now, but that change is fairly recent.
Are security groups/network ACLs (https://docs.aws.amazon.com/elasticloadbalancing/latest/network/load-balancer-troubleshooting.html#target-not-in-service) set to allow communication between the load balancer and pods?

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-backend-protocol: tcp
    service.beta.kubernetes.io/aws-load-balancer-type: nlb
  name: kong-proxy
  namespace: kong
spec:
  externalTrafficPolicy: Cluster```

Awesome. We have a ticket tracking to change the default to externalTrafficPolicy to Cluster. Buggy EKS implementation has caused more problems with the Local policy than it is worth.
Thank you for your patience.

I'd like to reopen this issue because the latest updates to https://bit.ly/k4k8s have created further problems. I have a 3-node cluster with an NLB provisioned to the kong-proxy service. In the latest update, Kong does not include an externalTrafficPolicy field, and so after I unsuccessfully deployed the latest manifest, I added back externalTrafficPolicy: Cluster because doing so had previously solved my issue. However, I have no idea what the issue could be this time around.
Kong Ingress Controller Version: kong/kubernetes-ingress-controller:1.3
Kubernetes Version: Client Version: version.Info{Major:"1", Minor:"21", GitVersion:"v1.21.1", GitCommit:"5e58841cce77d4bc13713ad2b91fa0d961e69192", GitTreeState:"clean", BuildDate:"2021-05-12T14:18:45Z", GoVersion:"go1.16.4", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"20+", GitVersion:"v1.20.7-eks-d88609", GitCommit:"d886092805d5cc3a47ed5cf0c43de38ce442dfcb", GitTreeState:"clean", BuildDate:"2021-07-31T00:29:12Z", GoVersion:"go1.15.12", Compiler:"gc", Platform:"linux/amd64"}
Environment: AWS GovCloud, AMI 1.20.4-20210826
Additional Details:

platform version: eks.2
ingress controller logs: time="2021-09-08T18:28:46Z" level=info msg="no configuration change, skipping sync to kong" component=controller
ingress controller proxy logs: 2021/09/08 18:18:39 [notice] 22#0: *6 [lua] init.lua:260: purge(): [DB cache] purging (local) cache, client: <IP_ADDRESS>, server: kong_admin, request: "POST /config?check_hash=1 HTTP/1.1", host: "<IP_ADDRESS>:8444"
<IP_ADDRESS> - - [08/Sep/2021:18:18:40 +0000] "POST /config?check_hash=1 HTTP/1.1" 201 6108 "-" "Go-http-client/1.1"

Per comment edit:

It seems that the problem was my organization's network security policies that were restricting me from executing the actions "authorizesecurityingress" and "createsecuritygroup". After those permissions were added, I was no longer getting any 403 errors in the kong-proxy logs. This can be closed.

re-closing.


bug:  LSP autostart

Did you check docs and existing issues?

[X] I have read all the plugin docs
[X] I have searched the existing issues
[X] I have searched the existing issues of plugins related to this issue

Neovim version (nvim -v)
NVIM v0.10.1
Operating system/version
MacOS 14.6.1
Describe the bug
Is there a way to get the lsp to autostart without having to run the game?
Steps To Reproduce
i read the documentation and dont see a way to autoattach the lsp on lua file load.
Expected Behavior
Provide way to attach to lua_ls

Hi, sorry for the late response.

Is there a way to autostart the LSP without running the game?

Yes, you can load love2d.nvim on the VeryLazy event instead of waiting for the first use of the LoveRun command. If you're using lazy.nvim, you can update your configuration like this:
{
  "S1M0N38/love2d.nvim",
  event = "VeryLazy",
  opts = {},
}


python37Packages.pg8000: 1.16.6 -> 1.17.0

Automatic update generated by nixpkgs-update tools. This update was made based on information from passthru.updateScript.
meta.description for python37Packages.pg8000 is: "PostgreSQL interface library, for asyncio"
meta.homepage for python37Packages.pg8000 is: "https://github.com/tlocke/pg8000"
meta.changelog for python37Packages.pg8000 is: ""
Updates performed

Ran passthru.UpdateScript

To inspect upstream changes
Impact

Checks done (click to expand)

built on NixOS

Warning: a test defined in passthru.tests did not pass

0 of 0 passed binary check by having a zero exit code.

0 of 0 passed binary check by having the new version present in output.

found 1.17.0 with grep in /nix/store/69fpjfrf3v980z9lm5g7fm6gjmhvm75m-python3.7-pg8000-1.17.0

directory tree listing: https://gist.github.com/4d9b5339ca0959ed584e5816b52fc9a0

du listing: https://gist.github.com/5d96366dac3dcefad22daf8eac150cb9

Rebuild report (if merged into master) (click to expand)

21 total rebuild path(s)

6 package rebuild(s)

6 x86_64-linux rebuild(s)
6 i686-linux rebuild(s)
3 x86_64-darwin rebuild(s)
6 aarch64-linux rebuild(s)

First fifty rebuilds by attrpath
python37Packages.pg8000
python37Packages.testing-postgresql
python38Packages.pg8000
python38Packages.testing-postgresql
python39Packages.pg8000
python39Packages.testing-postgresql

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/69fpjfrf3v980z9lm5g7fm6gjmhvm75m-python3.7-pg8000-1.17.0 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A python37Packages.pg8000 https://github.com/r-ryantm/nixpkgs/archive/5f1696c1687190aec69d4d3736f140af23c17ea0.tar.gz

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/69fpjfrf3v980z9lm5g7fm6gjmhvm75m-python3.7-pg8000-1.17.0
ls -la /nix/store/69fpjfrf3v980z9lm5g7fm6gjmhvm75m-python3.7-pg8000-1.17.0/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

6 packages built:

python37Packages.pg8000
python37Packages.testing-postgresql
python38Packages.pg8000
python38Packages.testing-postgresql
python39Packages.pg8000
python39Packages.testing-postgresql

Maintainer pings
cc @domenkozar for testing.

Successor #127764


update WMCore from CRIC legacy APIs to current/native APIs

Stefano points out the WMCore, i.e.
https://github.com/dmwm/WMCore/blob/fbaabff7b705eafaf11c558bf1f04c95dece9f57/src/python/WMCore/Services/CRIC/CRIC.py
uses the SiteDB legacy APIs. We think it's time to switch to the native/current CRIC APIs.
Impact of the new feature
CRAB, WMAgent/production, and any other user of the WMCore API
Is your feature request related to a problem? Please describe.
Gives us the image of living in the past
Describe the solution you'd like
service using the native/current CRIC APIs
Describe alternatives you've considered
relying on support of the old/legacy API
Additional context
n/a

@stlammel Stephan, thank you for creating this ticket. Would you - or @belforte - know what are the native CRIC APIs?
If they provide the same output in the same format, then it's a trivial update. Otherwise we have to make further changes to deal with the new data structure. Thanks

Hallo Alan,
the site API has both list of sites and associated Storage Units of each site.
https://cms-cric.cern.ch/api/cms/site/query/?json
It's one JSON but of course differently structured. Maybe a review of the CRIC.py and including Panos in the discussions would be good.
Thanks,

Stephan

this is not simply for elegance and stop using SiteDB objects which nobody wants to learn about anymore, staring with PhedexNoneName!
The problem is that now we have a couple sites at GRIF which have an RSE but no PSN and things break in various ways, even after @Panos512 tried to patch the SiteDB compatibility API. That API can maybe be hacked more and more, but does not look sustainable in "years to come" to me.

Thinking more about this, this is even more complicated: CRIC only knows about RSEs that have an associated processing site. So, ..., that module needs to query CRIC for a site list, Rucio for the RSE list, and CRIC for any processing-storage associations.
(Otherwise people cannot send CRAB output to a Tier-3 that has only storage and no CE.)

Stephan

CRIC only knows about RSEs that have an associated processing site

this seems a shortcoming regardless of CRAB StageOut. We can surely query Rucio from CRAB instedad of CRIC, but sort of defeats the idea of having a resource catalog


Fixes 3459: Update zero state component

Summary

Adds the following panel with two cards to the zerostate view.

Feedback on copy desired and welcome! (@swadeley)

Testing steps

Click the buttons for each card, ensure navigation works as intended.

https://issues.redhat.com/browse/HMS-3459

/retest

/retest

if i click Browse Popular repositories from the zero state, it looks like then clicking on Your repositories navigates back to the zero state. is that expected? i don't see this behavior when clicking Browse Red Hat repositories or Add repositories now from the zero state (meaning i can navigate anywhere without being routed back to zero state)

This should be more than safe to merge @swadeley if you'd like to test against stage.

/retest

/retest


Question about RotatE new evaluation.

In RotatE new evaluation, you change the order of scores before sorting. So the key problem becomes the sort algorithm used in torch.argsort should be stable. I am not sure about this. But I google it. Seems like it's not. So I think it would be a problem. Maybe we should change to np.argsort(xxx, kind='stable')?
Try to this:

import torch
a = torch.tensor(
[[ 0., 3.],
[ 2., 3.],
[ 2., 2.],
[10., 2.],
[ 0., 2.],
[ 6., 2.],
[10., 1.],
[ 2., 1.],
[ 0., 1.],
[ 6., 1.],
[10., 0.],
[12., 0.]]
)
print(a[torch.argsort(a[:, 0])])

Output:

tensor([[ 0.,  3.],
[ 0.,  2.],
[ 0.,  1.],
[ 2.,  1.],
[ 2.,  2.],
[ 2.,  3.],
[ 6.,  1.],
[ 6.,  2.],
[10.,  1.],
[10.,  2.],
[10.,  0.],
[12.,  0.]])

The default setting of np.argsort is 'quicksort', which won't change the position of the first element if it's already one of the smallest elements. But I agree that 'stable' might be better.


[BEAM-2645] Define the display data model type

Use a URN + value system with an optional label allowing for 'dynamic' types to be specified and labelled with a set of well known types.
Fix-up existing Java usage.

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

[ ] Choose reviewer(s) and mention them in a comment (R: @username).
[ ] Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
[ ] Update CHANGES.md with noteworthy changes.
[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

R: @robertwb @dpmills

Run Java PreCommit

This is not used in Python or Go.


Remove console log -- sorry :(

@nzambello @giuliaghisini @tiberiuichim @avoinea @pnicolli @datakurre 5.2.0 have a nasty console.log dangling around :( my bad, sorry! I will release 5.2.1 right away.


The use of selected  segment is wrong

in User events, there are 2 segments received and performed. when you tap on received it shows the performed ones instead.and vice versa.
I debugged this line https://github.com/khoren93/SwiftHub/blob/master/SwiftHub/Modules/Events/EventsViewModel.swift#L138 the segment returns received when you tap on Performed segment.
I added  another segment, so we have now 3 segments, it returns wrong segment index each time.
Same for Issues.

Nice !
Btw I played with the code to use Collectionview instead of tableview. All seems good but one thing is that when data is empty I can't seem to refresh, altough it is wokring well with tableview. do you know the reason?

Try to set alwaysBounceVertical = true in your collection view.


Schema parsing - Permit opaque key-values in Property schema

Currently, this schema is valid according to https://www.jsonschemavalidator.net/
{
    "$id": "https://www.github.com/schemas/robfig",
    "properties": {
        "name": {
            "type": "string",
            "opaque": "something"
        }
    }
}

But it can not be unmarshaled, receiving this error:
error unmarshaling properties from json: error unmarshaling opaque from json: json: cannot unmarshal string into Go value of type jsonschema._schema

That is due to this code, which assumes any extra key must be a Definition:
	for prop, rawmsg := range valprops {
		var val Validator
		if mk, ok := DefaultValidators[prop]; ok {
			val = mk()
		} else {
			switch prop {
			// skip any already-parsed props
			case "$schema", "$id", "title", "description", "default", "examples", "readOnly", "writeOnly", "$comment", "$ref", "definitions", "format":
				continue
			default:
				// assume non-specified props are "extra definitions"
				if sch.extraDefinitions == nil {
					sch.extraDefinitions = Definitions{}
				}
				s := new(Schema)
				if err := json.Unmarshal(rawmsg, s); err != nil {
					return fmt.Errorf("error unmarshaling %s from json: %s", prop, err.Error())
				}
				sch.extraDefinitions[prop] = s
				continue
			}
		}
		if err := json.Unmarshal(rawmsg, val); err != nil {
			return fmt.Errorf("error unmarshaling %s from json: %s", prop, err.Error())
		}
		sch.Validators[prop] = val
	}

The relevant portion of the spec says that definitions must be collected under a "definitions" key
https://json-schema.org/latest/json-schema-validation.html#rfc.section.9
So, it seems like the the code should be updated to pull out just the "definitions" key, and then apply the "parse the values of each key under that as a Schema".
If you agree, I will add unit tests for this case and make a pull request.

@robfig That's not quite what that part of the spec means.  The definitions key is the standard place to put definitions, but you are not forced to do so.
The problem here is that the spec is clear that implementations SHOULD ignore keywords they do not support.
I had thought that there was a MUST NOT error, but I guess not.  I might add that in the next draft.  I think it is SHOULD be ignored rather than MUST because it would be perfectly valid for an implementation to treat such keywords as extra data like title or description.  Possibly unexpected or unwanted, but it wouldn't break things.

So you're saying that additional definitions are not required to be collected under a "definitions" keyword, but the example from the link:
{
    "type": "array",
    "items": { "$ref": "#/definitions/positiveInteger" },
    "definitions": {
        "positiveInteger": {
            "type": "integer",
            "exclusiveMinimum": 0
        }
    }
}

could equivalently be
{
    "type": "array",
    "items": { "$ref": "#/definitions/positiveInteger" },
    "positiveInteger": {
        "type": "integer",
        "exclusiveMinimum": 0
    }
}

?
I'm having trouble reconciling that meaning with the linked section on json-schema.org, but it seems like you know what you're talking about.
In that case, I suppose the right way to allow opaque key/values at this level in the document would be to continue to attempt to parse the values as a definition, but ignore any errors that result instead of treating them as fatal.
How does that sound?

it seems like you know what you're talking about.

Well, I'm the "handrews" in draft-handrews-json-schema-01 😝
The example you gave saying that those things would be equivalent is not correct, though.
$ref takes a URI, and URIs like #/definitions/positiveInteger are JSON Pointers.  So that form of $ref must match the JSON object structure.  $ref also MUST point to a JSON Schema (and not, say, a string or number or array, or an object with some other semantics).
I'm not sure why you are focused on definitions here, to be honest.  Adding a custom extension keyword such as opaque really has nothing to do with where definitions live or how you reference them.

Thanks for filing this @robfig. Totally agreed the error you've encountered is a bug that needs fixing.
I think @handrews has articulated the we should try to tackle here:

The problem here is that the spec is clear that implementations SHOULD ignore keywords they do not support.

I think you're totally right to point to the poor assumption this lib currently makes in assuming everything is a definition. We should be doing what the spec says and ignore unsupported keywords.
However, one behavior I'd like to keep is not just ignoring unrecognized keywords, but also giving them back if a user were to ever json.Marshal a jsonschema.RootSchema. Put another way, the de-serialization process shouldn't be semantically lossy. We would obviously clobber whitesapce from input data, but values encoded should also be decoded.
Thanks so much for your PR, the unit test you've provided is great, but I agree the solution needs to be a little more robust. I think we should refactor the lib to carry deserialize unknown values to opaque values instead of "extra definitions", and re-add any opaque values on json.Marshal. Sound good?

The mismatch in the second example between $ref and the location of the definition was unintentional. I meant it to be this:
{
    "type": "array",
    "items": { "$ref": "#/positiveInteger" },
    "positiveInteger": {
        "type": "integer",
        "exclusiveMinimum": 0
    }
}

I'm focused on definitions here because that's what this implementation is treating my opaque keyword as, instead of ignoring it as I would prefer. In your last comment, you said that it was allowed to do so.. that might be true, but it's also allowed to not fail to parse a schema with an opaque keyword, and that's arguably the more helpful behavior.

@robfig I'd be happy to work with you on the change, I can write up a rough outline of a PR we'd accept if you're ok with implementing the change

hahah looks like we came to the same conclusion @robfig 😄

Hey all,
i was provided with a json schema from another customer which is relatively big and has severe typos in several locations, e.g."$comments". Those are running into the here mentioned definitions:
error unmarshaling properties from json: error unmarshaling $comments from json: json: cannot unmarshal string into Go value of type jsonschema._schema

The json schema should be fixed but to get something fast of the ground it would be handy to have some kind of relaxed syntax check option.
Thanks for your work and the jsonschema lib in any case!

I'm also blocked on this issue for a project. Can I help move things forward somehow?

@b5 for some reason I didn't notice this part before:

However, one behavior I'd like to keep is not just ignoring unrecognized keywords, but also giving them back if a user were to ever json.Marshal a jsonschema.RootSchema. Put another way, the de-serialization process shouldn't be semantically lossy. We would obviously clobber whitesapce from input data, but values encoded should also be decoded.

Now that draft 2019-09 (formerly known as draft-08 but we switched to dates for meta-schema versioning) is out, you should take a look at the recommended output formats, specifically for collecting annotations (the output format is also for error reporting, and I think most of the examples focus on that- we should add more annotation examples).
We considered saying that unknown keywords SHOULD be collected as annotations, but annotation-collecting is an optional feature (we didn't want to suddenly make all existing validators non-conforming by requiring a whole new concept to be implemented).  But that's basically what it sounds like you want to do, so I'd recommend taking a look at how 2019-09 specifies that should work.


restarting a peer can lead to permanent global non-convergence and connection churn

On a number of occasions we have seen log files that show connections shutting down with Multiple peers found with same name: ... and Connection appears to be with different version of a peer we already know of with associated significant levels of connection churn.
To date we considered these to be transitory conditions, caused by delays in peers detecting that one of their connections is stale, and thus still believing they are connected to the old incarnation of a restarted peer. Heartbeat timeouts should resolve that in less than ~1 minute.
However, users report that they have seen these errors persist for far longer than 1 minute.

I have a somewhat vague theory...
For a restarted peer to be permanently "accepted" into a network, all peers in that network must no longer hold a reference to the old incarnation. That requires a) all peers that were connected to that peer to detect the loss of connectivity to that peer, and b) for that information to propagate to all other peers. Furthermore, all this needs to happen before any peer accepts the new incarnation of a peer. Otherwise, in effect, the "old incarnation of peer has gone away" and "here is a (new incarnation of a) peer" facts race around the network in the topology gossip and whenever they collide end up dropping the gossiping connection.
I haven't totally convinced myself that convergence is impossible in some such circumstances, but it's certainly non-trivial to proof that it is.

Here is a proposed fix:

we get rid of all UID checks. In particular a peer will no longer drop the connection when discovering that

that the remote peer of a new connection has a different UID from what it currently knows, and
received topology gossip refers to a peer with a UID different from what it currently knows.

when updating peer information from topology gossip, we set the UID to that which was received.
when topology gossip contains information about ourself, if the received version number is greater than ours we set ours to that

The underlying idea is that knowledge in the network of an old incarnation of a peer will be replaced by information about the incarnation without requiring an intermediate step where peers are able to 'gc' the old incarnation.

Hmm. I think this means we can get rid of the UID completely.

when topology gossip contains information about ourself, if the received version number is greater than ours we set ours to that +1

There is an edge case here that prevents convergence in the absence of change: if the network has knowledge of an old incarnation at version V, and the new incarnation is also at version V, then the latter information will not supplant the former. That only happens when there is a further change to the peer, bumping the version to V+1.
I can think of two ways of dealing with that:

when topology gossip contains information about ourself, if the received version number is greater than ours we set ours to that +1; if it is equal to ours and the received UID is different from ours we increment our version

when topology gossip contains information about ourself, if the received version number is greater than ours we set ours to that +1; if it is equal to ours we compare the contents (i.e. the list of connections and short id) and if they differ we increment our version number

The first prevents us from getting rid of UIDs. The second is potentially expensive, especially considering that we will nearly always hit the comparison case while the local peer is not changing. We could eliminate the cost by adding a hash of the content, but arguably that is no better than just keeping UIDs.
And I reckon we also need to introduce the following tie-breaker:

when receiving information from topology gossip about a peer, if the versions are the same we update our information if the received UID is greater than ours

That way there is convergence in the "same version but different UID" case, albeit possibly to the "wrong" state, i.e. using information from an old incarnation, but this will only be a temporary aberration.

Backward compatibility is an issue here, even if we keep UIDs. The problem is that, as explained, current routers very much depend on the entire network forgetting about a peer first before accept a new incarnation of a peer. And the disconnects on UID mismatch rather help in creating such a situation. New routers wouldn't do that.
One possibility here is to keep the disconnecting behaviour for direct connections, i.e. the Connection appears to be with different version of a peer we already know of case. This would increase the chances substantially that the old incarnation will be "flushed" from the topology across the network. But it doesn't guarantee that. Indeed this whole issue exist precisely because we cannot guarantee that even with all the existing disconnection behaviour.
We could alternatively/additionally advise users to "wait a while" when restarting a peer.

I cannot manage to come up with a scenario where the old peer is not forgotten.
For example, suppose we have peers A, B, C, D, connected in a ring, i.e. A<->B<->C<->D<->E and A<->E. At the start of this analysis, all peers know about all other peers.

C is restarted, with same peer name and different UID.  Call this C2.
B passes gossip to A containing info on C2 and B<->C2 connection.
connection from A to B is instantly dropped as C2 clashes with C.

A does not garbage-collect C as it still sees a connection via E.

D passes gossip to E containing info on C2.
connection from D to E is dropped as C2 clashes with C.

E does not garbage-collect C as it still sees a connection via A.
However, A will now gossip to E to say that its connection to B is gone, and once that happens, E will garbage-collect C and accept the new C2.
Now A will drop the connection to E because E's knowledge of C2 clashes, and so A has no connections to anywhere and will garbage-collect everything.
We see peer version numbers in the millions in logs, so connections were being added and removed very frequently.  #1496 should slow this down, and perhaps allow us to understand the real problem underlying this report.

A will now gossip to E to say that its connection to B is gone

Not if it since has been re-established.

#1496 should slow this down, and perhaps allow us to understand the real problem underlying this report.

#1496 will help. It may even make the problem go away. However, the scheme we have for dealing with re-starting peers is not obviously convergent, since it in effect relies on the transmission of events rather than states. And it couples the topology gossip in very intricate ways to connection lifecycle, namely closing connections on certain error conditions is required for convergence.
My proposal fix addresses that head on.

A will now gossip to E to say that its connection to B is gone

Not if it since has been re-established.

How can this happen?  As long as B is telling A about C2, A will disconnect from B.

How can this happen? As long as B is telling A about C2, A will disconnect from B.

The gossip is received some time after connection establishment.

OK, but some time after that A will tell E that A->B is gone, or A will try to tell E about C2 and E will drop E->A.  Either way, E will lose all connectivity to C and garbage-collect it.

but some time after that A will tell E that A->B is gone

Not if the connection has been re-established. If you have two peers who continuously attempt to connect each other and succeed, only for the connection to be dropped due to some gossip, it is possible that the "disconnected" state is never propagated to other peers.

My broader point though is that gossip convergence should not depend on careful orchestration of the underlay connectivity. That's just asking for trouble.

OK, looking again at the logs I see that it is typical for the collection to last 10-40ms before being torn down, while the delay before reconnection is perhaps 2-10ms.  With many peers, the chance that at least one of them claims to have a path to the old peer is increased.
FWIW, I couldn't get this from the description:

Otherwise, in effect, the "old incarnation of peer has gone away" and "here is a (new incarnation of a) peer" facts race around the network in the topology gossip

What about:
Otherwise, in effect, the "this connection to an old peer has gone away" and "I have a new connection which I still think leads to the old peer" facts race around the network in the topology gossip
?
And

all this needs to happen before any peer accepts the new incarnation of a peer

is disproved by my first attempt at a counter-example.

Otherwise, in effect, the "this connection to an old peer has gone away" and "I have a new connection which I still think leads to the old peer" facts race around the network in the topology gossip

The two facts I mentioned are the new facts. They both race around the network, together with gossip about the old state, i.e. the fact you mention.

is disproved by my first attempt at a counter-example

It isn't.

re backward compatibility

One possibility here is to keep the disconnecting behaviour for direct connections, i.e. the Connection appears to be with different version of a peer we already know of case. This would increase the chances substantially that the old incarnation will be "flushed" from the topology across the network.

@bboreham and I are not convinced this really helps, and it delays acceptance of new peers.
There is an interesting question here though... What should we do when accepting such a connection? In particular, should we update the UID? That would be oddly non-monotonic. We could have existing connections to the old incarnation that we haven't noticed dropping yet, the peer might be bouncing rapidly, etc. If we update the UID we hold for the peer, it would be down to luck to end up with the most recent one. Also, the version would be bogus.

I have finally managed to reproduce this. Sort of.
First apply the following patch...
diff --git a/router/gossip_channel.go b/router/gossip_channel.go
index 11e8e8a..6eaba9a 100644
--- a/router/gossip_channel.go
+++ b/router/gossip_channel.go
@@ -4,7 +4,10 @@ import (
        "bytes"
        "encoding/gob"
        "fmt"
+       "strings"
        "sync"
+
+       "github.com/weaveworks/weave/common/debug"
 )

 type GossipChannel struct {
@@ -156,6 +159,13 @@ func (c *GossipChannel) sendDown(conn Connection, data GossipData) {
        sender, found := c.senders[conn]
        if !found {
                sender = NewGossipSender(func(pending GossipData) {
+                       if c.name == "topology" && strings.Contains(c.ourself.NickName, "host1") {
+                               c.log("DROPPING gossip to", conn.Remote().String())
+                               return
+                       }
+                       if c.name == "topology" && strings.Contains(conn.Remote().NickName, "host1") {
+                               debug.Break("[gossip %s]: sending to %s", c.name, conn.Remote())
+                       }
                        for _, msg := range pending.Encode() {
                                if len(msg) > maxFeasibleMessageLen {
                                        panic(fmt.Sprintf("Gossip message too large: len=%d bytes; on channel '%s' from %+v", len(msg), c.name, pending))
@@ -213,7 +223,9 @@ func (c *GossipChannel) relayBroadcast(srcName PeerName, update GossipData) erro
        }
        broadcaster, found := c.broadcasters[srcName]
        if !found {
-               broadcaster = NewGossipSender(func(pending GossipData) { c.sendBroadcast(srcName, pending) })
+               broadcaster = NewGossipSender(func(pending GossipData) {
+                       c.sendBroadcast(srcName, pending)
+               })
                c.broadcasters[srcName] = broadcaster
        }
        broadcaster.Send(update)
@@ -231,6 +243,10 @@ func (c *GossipChannel) sendBroadcast(srcName PeerName, update GossipData) {
                protocolMsg := ProtocolMsg{ProtocolGossipBroadcast, GobEncode(c.name, srcName, msg)}
                // FIXME a single blocked connection can stall us
                for _, conn := range connections {
+                       if c.name == "topology" && strings.Contains(conn.Remote().NickName, "host1") {
+                               c.log("DROPPING broadcast to", conn.Remote().String(), "from", srcName)
+                               continue
+                       }
                        conn.(ProtocolSender).SendProtocolMsg(protocolMsg)
                }
        }

This also requires weave/common/debug/debug.go:

package debug

import (
	"fmt"
	"os"
	"os/signal"
	"syscall"
)

type breakpoint struct {
	format string
	args   []interface{}
	resume chan<- struct{}
}

type debugger struct {
	enabled     bool
	breakpoints []breakpoint
	suspend     chan<- breakpoint
}

var d debugger

func init() {
	suspend := make(chan breakpoint, 0)
	d = debugger{false, nil, suspend}
	sigs := make(chan os.Signal, 1)
	signal.Notify(sigs, syscall.SIGUSR1, syscall.SIGUSR2)
	go func() {
		for {
			select {
			case sig := <-sigs:
				switch sig {
				case syscall.SIGUSR1:
					d.enabled = !d.enabled
					if !d.enabled {
						fmt.Fprintln(os.Stderr, "DEBUG: breakpoints disabled")
						resume()
					} else {
						fmt.Fprintln(os.Stderr, "DEBUG: breakpoints enabled")
					}
				case syscall.SIGUSR2:
					resume()
				}
			case bp := <-suspend:
				if d.enabled {
					fmt.Fprintf(os.Stderr, "BREAK "+bp.format+"\n", bp.args...)
					d.breakpoints = append(d.breakpoints, bp)
				} else {
					bp.resume <- struct{}{}
				}
			}
		}
	}()
}

func resume() {
	for _, bp := range d.breakpoints {
		fmt.Fprintf(os.Stderr, "RESUME "+bp.format+"\n", bp.args...)
		bp.resume <- struct{}{}
	}
	d.breakpoints = nil
}

func Break(format string, args ...interface{}) {
	resume := make(chan struct{}, 0)
	d.suspend <- breakpoint{format, args, resume}
	<-resume
}

Then establish the following topology using your desktop machine and our three vagrant VMs:
        host2
       /  |  \
desktop   |   host1
       \  |  /
        host3

I do this by

host1: weave launch --no-discovery
host2: weave launch --no-discovery <IP_ADDRESS>
host3: weave launch --no-discovery <IP_ADDRESS> <IP_ADDRESS>
desktop: weave launch --no-discovery <IP_ADDRESS> <IP_ADDRESS>

The patch above

drops topology gossip from host1
drops topology broadcast to host1
allows us to pause the topology gossip to host1

Once everything is settled (check with weave status on each node, to see 4 peers (with 10 established connections))...

host2: WPID=$(docker inspect -f '{{.State.Pid}}' weave)
host3: WPID=$(docker inspect -f '{{.State.Pid}}' weave)
host2: sudo kill -USR1 $WPID
host3: sudo kill -USR1 $WPID

This pauses topology gossip from host2/3 to host1.
Next we wait for the "breakpoints" to hit - there will be messages like
BREAK [gossip topology]: sending to be:84:4f:13:85:c0(host1)

in the host2/3 weave docker container logs.
Once that has happened we know that any subsequent gossip will be merged into the existing gossip data held by the GossipSenders.
We now restart weave on our desktop:

desktop: weave stop
desktop: weave launch --no-discovery <IP_ADDRESS> <IP_ADDRESS>

Wait for a while for things to settle.  Eventually - it takes a while because of the continued dropping of topology gossip from host1 and topology broadcast to host1 - we see the connection between the two getting dropped with something like
ERRO: 2015/11/05 11:10:03.537339 ->[<IP_ADDRESS>:37340|1e:b9:3c:eb:14:b7(host2)]: connection shutting down due to error: Multiple peers found with same name: 0a:f7:0d:d0:03:df

Eventually host2 will reconnect to host1, and, after a while, the same error occurs. And so on, indefinitely.

The test demonstrates that some extreme scheduling can result in indefinite non-convergence. Essentially host1 never forgets about the old incarnation of the desktop peer because it thinks it has a route to it via host3. The situation resolves itself when we re-enable the gossip from host3 to host1. That too will result in the connection getting dropped with the above error. But crucially at that point host1 no longer has any connection to either host2 or host3, hence no route to the desktop peer, and hence it garbage-collects it. Next time host2 or host3 connect to it, no conflict is detected.
Note though that because the conflict is detected some time after the connection is established - on receipt of some gossip - it is possible, I think, for the connections from host2 and host3 to host1 to always overlap, i.e. there always being a connection to at least one of them, which in turn prevents garbage-collection of the desktop peer from host1. This situation would have been much more likely pre 1.2.0, due to the lack of backoff on late connection failures.
The test also shows that just network disruption can at least cause prolonged, but temporary, non-convergence. The dropping and pausing of topology gossip/broadcast we introduce could equally just be indefinite queueing due to TCP buffers filling up. However, any such disruption would cause the connections to be terminated after at most heartbeat-interval * 2. And this in turn would cause host1 to garbage-collect the stale information about the desktop peer.


Expose the ExtensionData's keys as a separate collection

The current ExtensionData dictionary only allows values to be accessed if the key is known ahead of time. Unfortunately, due to how DotLiquid handles dictionaries, there isn't a way to loop through the keys. This PR will expose the extension data keys for use in templates.

@RicoSuter -- you may have more experience with the Drop and IIndexable interfaces, but creating a new collection was the only way I could figure out to maintain backward compatibility.

you may have more experience with the Drop and IIndexable interfaces, but creating a new collection was the only way I could figure out to maintain backward compatibility

You mean this is the only way to actually iterate the keys in a template?

You mean this is the only way to actually iterate the keys in a template?

Correct.

Once you approve this PR, how long does it take to flow back to the NuGet packages?

Why did you close/delete the PR?

I realized the enumeration wasn't needed for my scenario as I can just as easily collapse my annotations into a single known extension if I am able to get the annotations PR template in.


[release-4.15] fix(oauth): Do not proxy IBM Cloud IAM endpoints

What this PR does / why we need it:
Which issue(s) this PR fixes (optional, use fixes #<issue_number>(, fixes #<issue_number>, ...) format, where issue_number might be a GitHub issue, or a Jira story:
Fixes #
Checklist

[ ] Subject and description added to both, commit and PR.
[ ] Relevant issues have been referenced.
[ ] This change includes docs.
[ ] This change includes unit tests.

Details in this issue:
https://issues.redhat.com/browse/OCPBUGS-15168

/hold
these might not be needed if main auto syncs still


Instantiation flow - Design Doc

Goal: =<2 libraries to install, <5 LOC to connect to a service
What should the library look like?
E.g. what methods should be available on the Fluid module?
import Fluid from "@fluidframework/aqueduct"

How should we connect to a Fluid service?
getTinyliciousContainer() is not sufficient unless we want to import a driver from each service.
E.g.
import getTinyliciousContainer from "@tinylicious/loader";
import getRouterliciousContainer from "@routerlicious/loader";

const tinyContainer = await getTinyliciousContainer(...);
const r11sContainer = await getRouterliciousContainer(...);

Alternative:
import tinylicious from "@tinylicious/loader";
import r11s from "@routerlicious/loader";
import frs from "@FRS/loader";

const tinyContainer = await tinylicious.init(...);
const r11sContainer = await r11s.init(...);
const FRSContainer = await FRS.init(...);

if (newContainer) {
    container.resolve('/', {...initSchema})
} else {
    container.request('/')
}

Alternative:
import tinylicious from "@tinylicious/loader";
import r11s from "@routerlicious/loader";
import frs from "@FRS/loader";

const tinyContainer = await tinylicious.init(...);
const r11sContainer = await r11s.init(...);
const FRSContainer = await FRS.init(...);

if (newContainer) {
    container.resolve('/', {...initSchema})
} else {
    container.request('/')
}

would it make sense to pass the 'newContainer' into the container itself, so there's just a single call?
container.resolve('/', newContainer, {...initSchema})

oh i see the resolve vs request......would need to consolidate those into a single function that could do either based on the 'newContainer'

@ChumpChief, do you see this is being part of the work you're currently doing?

@ChumpChief, do you see this is being part of the work you're currently doing?

Potentially could be, I had not been looking at this part in particular yet.

Potentially could be, I had not been looking at this part in particular yet.

I'm not sure if this is the best place for this feedback, but I figured I'd start here. It's still a mystery to me why Fluid would require API consumers (especially at the Aqueduct level) to have to plumb together different data object/DDS types in factory methods. If I use a data object type, that should be enough to get it wired up into whatever ambient dependency injection/factory context is provided by default. Similarly, if I define a new data object type that uses various nested data object types, that should be as simple as defining a class that inherits from a shared base class and then using the nested data object types. Expecting the user to know to create a ContainerRuntimeFactoryWithDataStore and add various provider entries etc. is putting a lot of mental-model load on the user. I've been experimenting with Fluid for a couple months now and still don't know how to create a simple React app with a custom data object type that uses two or three different nested data object types. 😆


Connect your non-Azure machines to Microsoft Defender for Cloud

Connect your non-Azure machines to Microsoft Defender for Cloud

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 9f3ffb32-67ae-883d-c250-6d17fe6f529d
Version Independent ID: efaf3c8c-4f4a-0dec-a567-d7f27fbfcf44
Content: Connect your non-Azure machines to Microsoft Defender for Cloud
Content Source: articles/defender-for-cloud/quickstart-onboard-machines.md
Service: defender-for-cloud
GitHub Login: @bmansheim
Microsoft Alias: benmansheim

@lowfitcorp1
Thanks for your feedback! We will investigate and update as appropriate.

@lowfitcorp1
Could you please elaborate the query for better understanding. So, we can assist you further.


upgrade flake8

Our current version
https://github.com/pytorch/vision/blob/1d7d92cb10c18b750a86acd0d971d114c34a40c4/.pre-commit-config.yaml#L22
is quite outdated. Yesterdays release of importlib-metadata==5.0.0 breaks flake8==3.9.2 since it uses a now removed functionality.
Instead of upgrading flake8, we could also pin importlib-metadata and stick with our old flake8 version, but I see no point. LMK if you think different.

CI job in question: https://app.circleci.com/pipelines/github/pytorch/vision/20764/workflows/9f8c7f23-34ef-4b66-a47c-e363601ee629/jobs/1689172


grpc python service multiprocess cannot use the same UDS socket

What version of gRPC and what language are you using?
grpcio 1.64.0
What operating system (Linux, Windows,...) and version?
Linux
What runtime / compiler are you using (e.g. python version or version of gcc)
python 3.10
What did you do?
Start multiprocess of UDS like example here, below is my script,
"""The gRPC AsyncIO server for the UDS example."""

import asyncio
import logging
import multiprocessing
import os

import grpc
import helloworld_pb2
import helloworld_pb2_grpc

log = logging.getLogger(__name__)

class Greeter(helloworld_pb2_grpc.GreeterServicer):
    async def SayHello(
        self,
        request: helloworld_pb2.HelloRequest,
        context: grpc.aio.ServicerContext,
    ) -> helloworld_pb2.HelloReply:
        log.info(f'request={request}, pid={os.getpid()}')
        del request
        return helloworld_pb2.HelloReply(message=f"Hello to {context.peer()}!")

async def serve() -> None:
    uds_addresses = ["unix:helloworld.sock", "unix:///tmp/helloworld.sock"]
    server = grpc.aio.server()
    helloworld_pb2_grpc.add_GreeterServicer_to_server(Greeter(), server)
    for uds_address in uds_addresses:
        server.add_insecure_port(uds_address)
        logging.info("Server listening on: %s", uds_address)
    await server.start()
    await server.wait_for_termination()

def start_worker():
    asyncio.run(serve())

if __name__ == "__main__":
    logging.basicConfig(level=logging.INFO)
    workers = []

    def _start_worker():
        worker = multiprocessing.Process(target=start_worker)
        worker.start()
        log.info(f'start worker {worker.pid}')
        return worker

    for _ in range(3):
        workers.append(_start_worker())

Then start this server with multiple processes, and then run the client to send sample requests.
What did you expect to see?
All processes should be able to take requests, just like how multiprocessing is supported by grpc at here
What did you see instead?
Only one process takes request.
Make sure you include information that can help us debug (full error message, exception listing, stack trace, logs).
See TROUBLESHOOTING.md for how to diagnose problems better.
Anything else we should know about your project / environment?
We did some investigation that multiple process cannot reuse UDS file handler. We want to know whether the grpc team has some knowledge how this could be accomplished. Thank you!

@gillmylady  Can you provide more details, sample client / server code . to repro the issue.

Hey, thank you for taking a look!
I provided the sample server code in the description to start 3 processes. You can use other existing greeter-client code to send requests.
Then, in the server code console, you should see the process ID for each request, and find the process ID the same no matter how many requests you send. Let me know if you need something else.
Best,
Han
On Oct 28, 2024, at 8:34 AM, Sourabh Singh @.***> wrote:

@gillmyladyhttps://github.com/gillmylady Can you provide more details, sample client / server code . to repro the issue.
—
Reply to this email directly, view it on GitHubhttps://github.com/grpc/grpc/issues/37984#issuecomment-2441465110, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AEBY26R2N5KL3NQ5GOD7IZTZ5YVLRAVCNFSM6AAAAABQPRFPZWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDINBRGQ3DKMJRGA.
You are receiving this because you were mentioned.Message ID: @.***>

Hi @gillmylady, like you rightly said, the UDS socket file handler does not support the so.reuseport option. It is only supported by TCP ports, and hence if you don't have any specific reason to use UDS sockets, you can try using TCP sockets.
If you do have to use a UDS socket, you will have to do something like use a UDP socket in the parent process, and a proxy TCP socket with so.resuseport enabled that the child processes can use.

Thank you! I can take a try. Do your team know whether this approach is stable and reliable for production usage? Are unary/stream, sync/async working normally?
Best,
Han
On Nov 5, 2024, at 9:22 AM, Sreenithi Sridharan @.***> wrote:

Hi @gillmyladyhttps://github.com/gillmylady, like you rightly said, the UDS socket file handler does not support the so.reuseport option. It is only supported by TCP ports, and hence if you don't have any specific reason to use UDS sockets, you can try using TCP sockets.
If you do have to use a UDS socket, you will have to do something like use a UDP socket in the parent process, and a proxy TCP socket with so.resuseport enabled that the child processes can use.
—
Reply to this email directly, view it on GitHubhttps://github.com/grpc/grpc/issues/37984#issuecomment-2456647978, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AEBY26WUV2NGINSMUGY7JTDZ7CE3XAVCNFSM6AAAAABQPRFPZWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDINJWGY2DOOJXHA.
You are receiving this because you were mentioned.Message ID: @.***>

Hi @gillmylady, both TCP and UDS sockets are safe for production usage for all the use cases you mentioned. You can give it a try and see if it solves the problem


Alert box issue during filtering #6640

Issue Reference
#6640

@papnoisanjeev
It is showing two different-different alert boxes for the same warning when the customer is trying to filter the orders. please find the screenshot Here

@Sanjaybhattwebkul
Removed  one alert box here.

@papnoisanjeev
Still not working. please find the screenshot Here,
The alert box should not be shown if all the required fields are filled

Resolve conflicts.

@devansh-webkul
Conflicts resolved.


33B inference too slowly

When I load the 33B model by the method shown in below, it's too slow to generate a token. And per token is about 2.9s
tokenizer = AutoTokenizer.from_pretrained(checkpoint)

tokenizer.pad_token = tokenizer.eos_token

model = AutoModelForCausalLM.from_pretrained(checkpoint, trust_remote_code=True, torch_dtype=torch.float16, low_cpu_mem_usage=True, device_map="balanced")

have any solution about this?

I notice you are using device_map="balanced", setting it to "auto" might be helpful.
If you are able to load the model in one GPU, don't load it into multiple, since passing parameters across GPUs is really slow. You could try using as few GPUs as possible for model parallelism.
You could also try quantizing the model(4 bit models are around 1GB/Bparameters):
https://huggingface.co/blog/4bit-transformers-bitsandbytes
Also, loading model with use_flash_attention_2=True could speed it up.


Add Tee Log

Add T-Log; the ability to also sent Serial output to a telnet server, mqtt, syslog or a near real time updating webpage hosted on the esp/arduino.

@ArduinoBot tag laid down.


Drag'n'Drop nodes in scene to exported NodePath

Would be awesome if it was possible to drop nodes from Scene Dock to exported NodePath property in Inspector Dock, like in Unity (sorry for saying that xDDD)
In current behavior user has to click property, click Assign, then Scene Tree Dialog shows up and user has to again find and select his node and click OK.
With d'n'd it would be to Lock Inspector on script node (? or any other idea) then find his nodes in 2D/3D view drag them to property in inspector. Done.

That was literally the first thing I tried before doing it the "manual" way.

duplicate of #2505


MetaKTSP

When I switch to MetaKTSP tab, there is a default error message. This error message is similar to MetaPCA.

solved by Ark


TypeError: expected str, bytes or os.PathLike object, not NoneType

import tensorflow as tf
Traceback (most recent call last):
File "", line 1, in
File "/home/aswin/anaconda3/envs/tfgpu/lib/python3.7/site-packages/tensorflow/init.py", line 98, in
from tensorflow_core import *
File "/home/aswin/anaconda3/envs/tfgpu/lib/python3.7/site-packages/tensorflow_core/init.py", line 45, in
from . _api.v2 import compat
File "/home/aswin/anaconda3/envs/tfgpu/lib/python3.7/site-packages/tensorflow_core/_api/v2/compat/init.py", line 23, in
from . import v1
File "/home/aswin/anaconda3/envs/tfgpu/lib/python3.7/site-packages/tensorflow_core/_api/v2/compat/v1/init.py", line 666, in
[_module_util.get_parent_dir(estimator)] + _current_module.path)
File "/home/aswin/anaconda3/envs/tfgpu/lib/python3.7/site-packages/tensorflow_core/python/tools/module_util.py", line 24, in get_parent_dir
return os.path.abspath(os.path.join(os.path.dirname(module.file), ".."))
File "/home/aswin/anaconda3/envs/tfgpu/lib/python3.7/posixpath.py", line 156, in dirname
p = os.fspath(p)
TypeError: expected str, bytes or os.PathLike object, not NoneType

Can you please provide more context to the issue by filling the template thats being provided here. Thanks!

I have this same issue on Ubuntu 19.04. Only happens with venv install. Works with system Python.

ok, i removed the current venv and created new venv. It is working now.


Updates to the hitch tests

Added the .hitch directory to gitignore.
Added README specifically for running the tests.
Removed the jinja2 parameterization in the stub test (it's an advanced feature; probably not really suitable for a quickstart project).
Directly specifies the version of python to download, install and build the virtualenv with in engine.py - 2.7.10 if python 2 is specified by cookiecutter, 3.5.0 otherwise.
Upgraded hitch requirements (includes a lot of bug fixes).
Override the default settings with an environment variable so that it sends an actual email to HitchSMTP on port 1025 during tests rather than using the console back end.
Now uses unixpackage to install prerequisite packages. Despite ubuntu packages being specified in settings.yml, the system will automatically try and install the correct equivalent for the platform it is on when running the test for the first time. On Arch it uses pacman, Fedora it uses yum, Mac OS X it uses brew, etc.
There's finally a one step set up for tests.

It's currently been tested on Debian, Ubuntu, Arch and Mac OS X Mavericks.

@crdoconnor one question about unixpackage:
when I added install_os_dependencies.sh script in https://github.com/pydanny/cookiecutter-django/pull/200, was commented the question that it is not prudent run, by default,  apt-get install without --no-upgrade option, because, by default, if a package is already installed and there is an update for it in the repository, apt-get install also the updates it.
This can result in breakage of existing legacy applications
https://github.com/pydanny/cookiecutter-django/pull/200#discussion_r26436046

Yep, that's a bug I'd say.
It checks in groups so if everything in the group is installed it won't run apt-get, but it will if just one of the packages is missing. Which is bad. It should only run apt-get for the missing package.


new AudioContext().decodeAudioData(arrayBuffer)  throws no output device available

My machine is a server and the windows server 2012R2 operating system is installed. There is no sound card. An error occurs when executing this code。
s it necessary to install a sound card or is there any other alternative? I'm just doing audio synthesis and don't need to play it

Hey,
You should probably use an OfflineAudioContext then.

I've used OfflineAudioContext. But there are two problems:

OfflineAudioContext initialization requires 3 parameters (fixed values set by me： new OfflineAudioContext(2, 44100 *1, 44100).decodeAudioData(arrayBuffer)）.
The playback speed (source.playbackRate.value = webAudioSourceBean.speed) of the synthesized audio does not take effect.

My development machine is Windows 10 and has a sound card. The playback speed after synthesis is normal. But when deployed to a server without a sound card, the synthesized audio speed has no effect.

The playback speed (source.playbackRate.value = webAudioSourceBean.speed) of the synthesized audio does not take effect.

This is weird and looks like an issue on its own, but this small test works properly on my side:
import { AudioContext, OfflineAudioContext } from 'node-web-audio-api';
import fs from 'node:fs';

const offline = new OfflineAudioContext(1, 44100, 44100);
const buffer = await offline.decodeAudioData(fs.readFileSync('sample.wav').buffer);

const src = offline.createBufferSource();
src.buffer = buffer;
src.playbackRate.value = 2;
src.connect(offline.destination);
src.start(0);

const output = await offline.startRendering();

// playback output buffer
const context = new AudioContext();
const playback = context.createBufferSource();
playback.connect(context.destination);
playback.buffer = output;
playback.start();

Could you elaborate a bit more on what you are trying to achieve? Its not clear to me
In another direction, maybe you can try new AudioContext({ sinkId: 'none' })

My goal is to develop an audio editor with functions such as fade in and fade out, cropping, and speed changing. The development machine Windows 10 has a sound card and all functions work normally. Deploy to the server (windows server 2012 does not have a sound card, but the virtual audio device is enabled), using two methods: new AudioContext({ ).decodeAudioData(arrayBuffer); and new OfflineAudioContext(2, 44100 * 1, 44100).decodeAudioData(arrayBuffer) , the speed changing function is invalid.
But according to your method new AudioContext({ sinkId: 'none' }), both the development machine and the server seem to be successful. thank you so much

ok cool!


Pac4jScalaTemplateHelper with PlayCookieSessionStore returns no profile

the Pac4jScalaTemplateHelper seems to explicitly specify the PlayCacheSessionStore. I setup my app using the PlayCookieSessionStore and noticed that on calls to the following:
val cacheProf = pac4jScalaTemplateHelper.getCurrentProfile(request)

val webContext = new PlayWebContext(request, playSessionStore)
val profileManager = new ProfileManager[CommonProfile](webContext)
val cookieProf = toScala(profileManager.get(true))

cacheProf is None, but cookieProf contains the current profile.
I got stuck on this for a while, so figured at least some more docs on this point could help others!

I don't think we need more documentation here, it's a bug, we should retrieve the appropriate PlaySessionStore, whether it is a PlayCacheSessionStore or a PlayCookieSessionStore.
Feel free to submit a contribution. Thanks.

Fixed in the incoming v7.0.1 release.


bug: cannot detect CUDA if the user does not install nvcc

Summary
The target machine has CUDA 12.2 installed, but it does not have nvcc. The installer failed to detect it.
Reproduction steps

Run the run-llm.sh script on a RHEL 8.6 machine with CUDA 12.2
Failed to install the GGML plugin.

Screenshots
No response
Any logs you want to share for showing the specific issue
The command and error: (NOTE I believe it tried to find libopenblas because it cannot find CUDA.
# bash <(curl -sSfL 'https://code.flows.network/webhook/iwYN1SdN3AmPgR5ao5Gt/run-llm.sh')
Checking the operating system (macOS and Linux supported) ...

Checking prerequisites ...
'libopenblas' is required for wasi-nn plugin to run.

The nvcc tool is not found.
# nvcc --version
bash: nvcc: command not found...

But nvidia-smi shows that GPU is installed.
# nvidia-smi
Fri Jan 12 11:44:01 2024
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 535.129.03             Driver Version: 535.129.03   CUDA Version: 12.2     |
|-----------------------------------------+----------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |
|                                         |                      |               MIG M. |
|=========================================+======================+======================|
|   0  NVIDIA H100 PCIe               Off | 00000000:47:00.0 Off |                    0 |
| N/A   42C    P0              79W / 350W |  65143MiB / 81559MiB |      0%      Default |
|                                         |                      |             Disabled |
+-----------------------------------------+----------------------+----------------------+
|   1  NVIDIA H100 PCIe               Off | 00000000:59:00.0 Off |                    0 |
| N/A   44C    P0              83W / 350W |  62544MiB / 81559MiB |      0%      Default |
|                                         |                      |             Disabled |
+-----------------------------------------+----------------------+----------------------+
|   2  NVIDIA H100 PCIe               Off | 00000000:C2:00.0 Off |                    0 |
| N/A   44C    P0              79W / 350W |  62544MiB / 81559MiB |      0%      Default |
|                                         |                      |             Disabled |
+-----------------------------------------+----------------------+----------------------+
|   3  NVIDIA H100 PCIe               Off | 00000000:D5:00.0 Off |                    0 |
| N/A   46C    P0              86W / 350W |  62544MiB / 81559MiB |      0%      Default |
|                                         |                      |             Disabled |
+-----------------------------------------+----------------------+----------------------+

Model Information
N/A
Operating system information
RHEL 8.6
ARCH
amd64
CPU Information
Intel on HP machines
Memory Size
N/A
GPU Information
4x Nvidia H100 with CUDA 12.2
VRAM Size
4x 64GB

@juntao
Could you please check if nvcc is under /usr/local/cuda/bin/nvcc? Or does this machine not install CUDA-related tools except for the libraries?

Hmmm. Okay, then, I should try to parse the CUDA version from the output of nvidia-smi. Reminder: the Jetson Orin doesn't support nvidia-smi due to the hardware design, so we cannot directly replace the original nvcc way with nvidia-smi.

So, I guess we need to check both? 😅

Yup. We should check both. However, we may need to determine the most elegant way to check CUDA.

As others note, you can also check the contents of the version.txt (.json in recent CUDA versions) using (e.g., on Mac or Linux)

cat /usr/local/cuda/version.txt

From: https://stackoverflow.com/a/9730706
I am not sure if there is a VERSION file for this.

For this machine, it does not have /usr/local/cuda

The installer now supports nvidia-smi for version detection.


Add MLX Cache Limit setting for mlx_lm.generate and mlx_lm.server CLI

This PR is to address the MLX Cache concerns from https://github.com/ml-explore/mlx-examples/issues/724

Looks great! Do you mind running the pre-commit hook for formatting? After that we can merge.

Looks great! Do you mind running the pre-commit hook for formatting? After that we can merge.

Sorry about the lack of formatting, I will make sure to do that on any future additions to the repo.


GFT: update wpaa source and logic

Per this comment, logic for wpaa variable needed additional twists. This PR addresses both "todo" bullet points in the comment.
The following has been implemented:

[x] update wpaa template with correct source
[x] update wpaa staging table with correct columns
[x] filter wpaa for status in buffer intermediate table
[x] update export geometry for wpaa dataset (previously it was point geom, the new dataset is in polygon)

Note

Old and new wpaa datasets have different geometry types. If I updated latest version with new data in recipes, it would yield issues in other ongoing PRs related to GFT. Therefore, I pulled new wpaa data without updating latest and fixed the wpaa version in GFT recipe file to work on this PR. Once PR is done:

[x] run data library to update latest version for wpaa dataset in recipes
[x] remove commit that fixes wpaa version in GFT recipe

There is one failing pytest in dcpy/library. I didn't touch dcpy in this PR, so not sure what it is. The failure should be addressed in a different PR.

Successful run here

Sorry for the flaky test - for testing live endpoints for ingest, should probably move that to the "nightly build" action instead of PRs
Hopefully rerunning clears it up for now though

Sorry for the flaky test - for testing live endpoints for ingest, should probably move that to the "nightly build" action instead of PRs
Hopefully rerunning clears it up for now though

Yep, passes now!

Re-ran data library for wpaa to update latest version in recipes, removed commit for wpaa version in GFT recipe, re-ran build with these changes (it passes here).


runescape chat

chat messages are "corrupted" like this
voice lines are just "Voice"
text is yellow which perfectly matches the default Condition color, making it hard to see
possible improvements:
-fix glitch
-disable "Voice", unless you want to make it show subtitles (or at least generic engineer.voice.helpme w/e)
-let us change text color/font
Gotta say I actually love the idea of this feature, being able to see what people nearby are saying is great.

I object to changing the colour, really I wanted to use DispatchUserMessage so I don't have to filter stuff like (Voice) and has received achievement and all that out but alas

The "corrupted" text is because it's not a wide string I believe and I can probably fix it

All issues fixed


Form model doesn't update on autocomplete

@yaroslavert This is a bit of a vague (or no) description. Any chance you caould provide a plunkr demonstrating the potential bug ?

(autofill) when page loads

Yep, that's already being logged/tracked/discussed here: https://github.com/angular/angular.js/issues/1460

why do not you fix a bug


Add RFID cookie in QueryJwtFilter

Cookie must be add with httpOnly = false

Implemented in 0.2.0


[LIBCLOUD-833] and LIBCLOUD-834

Changes Title (replace this with a logical title for your changes)
Description
[LIBCLOUD-833] - The AWS China region requires V4 authentication. Set the auth version to v4 allowing the region to function.
LIBCLOUD-834 - With V4 auth enabled the upload_via_stream function does not work and fails with an "InvalidCreds" error. This work around disables the supports_s3_multipart_upload section allow files to be uploaded to V4 regions.
For more information on contributing, please see Contributing
section of our documentation.
Status
Replace this: describe the PR status. Examples:

ready for review

Checklist (tick everything that applies)

[X] Code linting (required, can be done after the PR checks)
[ ] Documentation
[ ] Tests
[ ] ICLA (required for bigger changes)

@lmorfitt I think you introduced some spacing errors that are failing the linter.

@tonybaloney - Thanks. Fixed it up.
Luke

k, LGTM 👍

@Kami - Thank you for the note.
I copied the existing code for S3APNE2Connection so incorrectly assumed this code was supported.


OceanPy

I am interested in help to develop code for Oceanography similar to what exists in the Astronomy community (especially astropy).  It appears to me the best place to do this is in the python-oceans repo, but I post this here to ask for guidance.
The first thing I'd be likely to do is port the units package from astropy and constants.  These have been very useful.
Please advise.

I do quite like the concept of a Quantity from astropy.units though.

Hi, lead developer of unyt here!
I'd love to have some oceanographers kick the tires and let me know what functionality you're missing. Please feel free to open issues with feature requests or bug reports after you've given it a try.

Ok, I could be persuaded to head to unyt, especially since @ngoldbaum is such a cool dude! :)
And what is this community doing about constants?
I'll add that what is in my oceanpy already has the astronomy factored out.
Was easy.

And while I have the mic, I want to emphasize how valuable astropy
has been to the astronomical community (especially the observers).
Having to deal with 5+ repositories just doesn't work for the 'average joe or jane'.

Let me go one step further and suggest a meeting of the minds, i.e.
a Zoom chat given our wide physical separation.   Not sure where
anyone on this thread lives, but I bet early California time will work.
How about this Friday November 1 at 8am PDT?

Ok, @ngoldbaum convinced me to use unyt.
This branch does so and includes a few custom Oceanography constants
added to the unyt name space:
https://github.com/profxj/oceanpy/blob/forces3/oceanpy/constants/oceandata2019.py
And here is a simple calculation too.
https://github.com/profxj/oceanpy/blob/forces3/docs/nb/Forces.ipynb
I should still love to have a broader discussion on what might best
benefit the Oceanography community.  But apparently that may need
to wait a bit.


Pin 5687: Verify tenant certified attributes for agreement creation

Closes PIN-5607
This route will check the tenant attributes against the selected descriptor when creating an agreement for itself or for a delegator

Do we need to implement this logic in the agreement process? I think we have everything we need already in the BFF, the call is named /agreements/verify but it doesn't seem to be related to agreements since an agreement does not yet exist (unless I am missing something).
I think we can do everything in the BFF, doing something similar to what @MalpenZibo did in #1218, but simpler:
1. We GET the delegation

2. We GET the delegator from the delegation

3. We use the hasCertifiedAttributes utility (already implemented in BFF) and return true if  it's true for at least one of these tenants

Let me know what you think. I would also consider renaming the call since it is not related to agreements but just to the verification of attributes given a descriptor and a delegation

As discussed in sync, it's better to keep the route implemented in the process rather than the BFF

A macro-doubt: looking at the UI where this call is used, the only thing that the user selects is a tenant. So, I would expect the body of the request to contain just eserviceId, descriptorId, and tenantId.
Then:
1. If the tenant selected is the request, we verify the attributes for that tenant, no delegation involved

2. otherwise, if the tenant is not the requester, we look for a consumer delegation that has the selected tenant as delegator and the requester as delegate, and if it exists we verify the attributes for the delegator - if it does not exist we return false

Or am I missing something?

In that page the frontend also has information on the delegationId but thinking about it, maybe it's better to implement as you suggest because there's one less parameter to check (I won't have to check that delegetionId is correct, we just retrieve the correct one if exists).
Thanks for the suggestion!


[YoutubeDL] Do not abandon playlist upon "format not available" error on video

The downloading of a playlist is currently terminated upon encountering the first "format not available" error for any single video item in the list.
This should be a 'warning' instead, so that the remainder of the playlist or channel can continue downloading.

Would make more sense to 1st check if there are formats to download, then act diff when there ain't, rather than the current one where non-existence is checked, then acted diff when there are.

@rautamiekka I agree, but I wanted to disturb the existing logic as little as possible in order to just fix the worst behavior.

This is expected behavior, -i is supposed to be used to not stop on errors.


Add ticket detail comments

The handler only shows the description for the first comment and not the latest comment. The request is to show the latest comment.

v0.1.0 of the adapter will now display comments in reverse chronological order similar to the Zendesk web UI.


Sdk 54: Java SDK: Add getter and setters hadoop2 and persistent security group

Adding missing getters and setters for hadoop2 and persistent security group

Please merge latest master and ensure that both of them are used only in API response and not directly created by the user in any way.

merged latest master


Newsletter clicks - update broken link not working for feedback links

Issue Summary
updating a broken link works for the main header in newsletters, but the feedback links do not get updated.
Steps to Reproduce

publish post that gets emailed
change post title and update "post url" on the post
both header link and the feedback links in the newsletter sent do not work resulting in a 404, but if you change the url manually in the browser they work
update the link by going to analytics on the post, then under newsletter clicks update the link to the corresponding post to the new link that was set in the "post url" on the post
in the same newsletter that was already emailed, clicking the header in the email will link to the new post url, but the feedback links at the bottom are not working in link to the old post url

Ghost Version
5.53.1 docker latest
Node.js Version
version included in 5.53.1 docker image
How did you install Ghost?
docker
Database type
MySQL 5.7
Browser & OS version
chrome 114.0.5735.134
Relevant log / error output
No response
Code of Conduct

[X] I agree to be friendly and polite to people in this repository

Hey there, thank you so much for the detailed bug report.
That does look like something that shouldn't happen! A PR to fix this issue would be very welcome 🙂

What is the functional usefulness of the ability to update the link in the analytics if it doesnt work?


use framework's hostname to initialize scheduler's UPID hostname

Frameworks do not quite expose themselves properly to the Mesos Master unless UPID.Host is set to match the listen address (conveniently FrameworkInfo.Hostname).

#69 fixes this, so closing


Add OCI artifact platform support

What type of PR is this?
/kind feature
What this PR does / why we need it:
This allows pushing and pulling multiple platforms as OCI artifacts.
Which issue(s) this PR fixes:

Refers to #1482
Does this PR have test?

Yes
Special notes for your reviewer:
None
Does this PR introduce a user-facing change?

Added support for platforms (`os[/arch][/variant][:os_version]`) when using seccomp OCI artifact profiles.

Codecov Report

Merging #1658 (c70b54d) into main (fa5d283) will increase coverage by 0.71%.
The diff coverage is 94.59%.

:exclamation: Current head c70b54d differs from pull request most recent head d206583. Consider uploading reports for the commit d206583 to get more accurate results

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1658      +/-   ##
==========================================
+ Coverage   48.62%   49.34%   +0.71%
==========================================
  Files          69       71       +2
  Lines        6788     6929     +141
==========================================
+ Hits         3301     3419     +118
- Misses       3356     3380      +24
+ Partials      131      130       -1

@kubernetes-sigs/security-profiles-operator-maintainers PTAL

Thank you @pjbgf, I addressed all the comments.


MacForge incorrectly reports itself uninstalled when another copy of MacForge.app is in ~/.Trash

Bug Report
Current Behavior
MacForge reports itself uninstalled when another copy of MacForge.app is in ~/.Trash.
To Reproduce
Steps to reproduce the behavior:

Download another copy of MacForge.app.
Trash the other copy to ~/.Trash.
Trash any other file, and MacForge will report itself uninstalled.

Expected behavior/code
MacForge should correctly not report itself uninstalled.
Screenshots

Environment

MacForge version(s): version 0.10.3 (1035)
OS version: macOS 10.14.6
Plugins: currently none running

Possible Solution
I think it would be fine if MacForge just wouldn't check if it's uninstalled.
Having an uninstall button in MacForge would be better. (No automatic detection)

Most likely I'll just add a check to see if the helper itself is in the trash. Even though this app is somewhat targeted at a power user (someone willing to disable SIP and AMFI) the goal is for it to be simple to use and most people would consider throwing an application in the trash uninstalling it.

0.11.1 will now only show that prompt if there is no longer a file where the helpers bundle was when it launched.


[#noissue] Upgrade dependencies

Codecov Report

Merging #6922 into master will decrease coverage by 0.00%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #6922      +/-   ##
==========================================
- Coverage   39.26%   39.26%   -0.01%
==========================================
  Files        3221     3221
  Lines       95272    95272
  Branches    11885    11885
==========================================
- Hits        37412    37406       -6
- Misses      54870    54877       +7
+ Partials     2990     2989       -1

Impacted Files
Coverage Δ

...p/pinpoint/rpc/client/WriteFailFutureListener.java
56.25% <0.00%> (-18.75%)
:arrow_down:

...vercorp/pinpoint/rpc/packet/ClientClosePacket.java
55.55% <0.00%> (-11.12%)
:arrow_down:

...java/com/navercorp/pinpoint/rpc/DefaultFuture.java
72.50% <0.00%> (-1.67%)
:arrow_down:

...orp/pinpoint/rpc/server/DefaultPinpointServer.java
79.22% <0.00%> (-1.06%)
:arrow_down:

...rc/main/java/com/navercorp/pinpoint/test/Item.java
66.66% <0.00%> (ø)

...pinpoint/profiler/monitor/DeadlockMonitorTask.java
19.76% <0.00%> (+1.16%)
:arrow_up:

...ollector/cluster/zookeeper/ZookeeperJobWorker.java
80.93% <0.00%> (+1.39%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8e31804...676f991. Read the comment docs.


Using WebSockets with ASP.NET MVC 5 on Azure

Hello,
I am using ASP.NET MVC 5 and have successfully tested Fleck on my local machine.  I deployed this application to an app service on Azure and I can't make a web socket connection.  Here is what I have on the Application_Start() :

void Application_Start(object sender, EventArgs e)
        {
            // Code that runs on application startup
            AreaRegistration.RegisterAllAreas();
            GlobalConfiguration.Configure(WebApiConfig.Register);

            // OPEN A WEBSOCKET CONNECTION.
            FleckLog.Level = LogLevel.Debug;
            var allSockets = new Dictionary<string, IWebSocketConnection>();
            var server = new WebSocketServer("ws://<IP_ADDRESS>:8080");
            server.Start(socket =>
            {
                socket.OnOpen = () =>
                {
                    //Console.WriteLine("Open!");
                    var guid = socket.ConnectionInfo.Path.Replace("/", "");
                    allSockets.Add(guid, socket);
                };
                socket.OnClose = () =>
                {
                    //Console.WriteLine("Close!");
                    var guid = socket.ConnectionInfo.Path.Replace("/", "");
                    allSockets.Remove(guid);
                };
                socket.OnMessage = message =>
                {
                    //Console.WriteLine(message);
                    allSockets.Values.ToList().ForEach(s => s.Send("Echo: Sending back the same message: " + message));
                };

                socket.OnBinary = (bytes) =>
                {
                    //Console.WriteLine("received binary");
                };
            });
        }

On the client side, here is what I have:
<script type="text/javascript">
        var ws = null;
        function createConnection() {
            ws = new WebSocket("ws://my-test-app.azurewebsites.net:8080");
            ws.binaryType = "blob";
            ws.onopen = function () {
                alert("About to send data");
                ws.send("Hello World"); // I WANT TO SEND THIS MESSAGE TO THE SERVER!!!!!!!!
                alert("Message sent!");
            };

            ws.onmessage = function (evt) {
                alert("About to receive data");
                var received_msg = evt.data;
                alert("Message received = "+received_msg);
            };
            ws.onclose = function () {
                // websocket is closed.
                alert("Connection is closed...");
            };
        };

        function sendMessage() {
            var txtBox = document.getElementById('messageTxt');
            var txtBox2 = document.getElementById('messageTxt2');

            const arr = [txtBox.value, txtBox2.value, 3, 5];

            //const array = new Float32Array(5);

            //for (var i = 0; i < array.length; ++i) {
            //    array[i] = i / 2;
            //}
            //console.log(array);
            ws.send(arr);
        }
    </script>

When trying to connect, I get the following error:
WebSocket connection to 'ws://my-test-app.azurewebsites.net:8080/' failed: Error in connection establishment: net::ERR_CONNECTION_TIMED_OUT
Any suggestions will be highly appreciated.
Thank you.

Most likely the ip the socket server is listening on, try changing the socket server ip to your domains ip address <IP_ADDRESS> instead of <IP_ADDRESS>