dep init freeze for hours

What version of Go (go version) and dep (git describe --tags) are you using?
go version go1.8 linux/amd64
dep latest version
What dep command did you run?
$ dep init -v
No Output
What did you expect to see?
The new files
*Gopkg.toml
*Gopkg.lock
*vendor(folder)
What did you see instead?
Nothing the command stuck there for hours and I have to press ctr+c to exit and have back the terminal.

hi! thanks for the issue. this is quite odd.
did you run this on an existing project, or something new? if the former, what if any tool were you using previously? is this reproducible?
no places in dep jump to mind where it could hang prior to generating any output. unless...@matjam, maybe a lockfile thing?
strace output would also probably help enormously to clear this up.

might be.
maybe I should add some stderr logging if its waiting for >60 seconds .. at least until you figure out the notification framework thing.

I have the same problem!

@matjam yeah, i'll relent and break my rule a bit more - we can add some print statements around the lock. probably fire one right away if it decides we have to wait, then...maybe one every 10s or so thereafter?

@sdboyer okay, I'll give you a PR with something that does that. I'll use 15s if only to be contrary.
We are just assuming that it's the locking thing, but, it's the most likely culprit.
I did do some research; the locking library is mostly correct in it's behaviour; there is a potential race there, but I've not been able to quantify what the real risk is of hitting it.
What is the addage? Race conditions are so rare that they always happen? Something like that.

ok, we have the warning message printing now when the lockfile is busy. @jonahfang, @cpapidas - could you update and see if you're still experiencing the indefinite hang?

@sdboyer It keep saying and still hang:
waiting for lockfile /go/pkg/dep/sm.lock: Lockfile created, but doesn't exist

@jonahfang OK, can we get some more details about the system?

What kernel/distribution?
Can you show us the mount options  (output of mount)?
Is this being run inside a container?
is there an existing sm.lock file?

@sdboyer Its this code in the lockfile package:
	// return value intentionally ignored, as ignoring it is part of the algorithm
	_ = os.Link(tmplock.Name(), name)

	fiTmp, err := os.Lstat(tmplock.Name())
	if err != nil {
		return err
	}
	fiLock, err := os.Lstat(name)
	if err != nil {
		// tell user that a retry would be a good idea
		if os.IsNotExist(err) {
			return ErrNotExist
		}
		return err
	}

So it's failing to link.
@nightlyone any ideas?

@jonahfang also, can you try the following
$ go get github.com/nightlyone/lockfile
$ cd $GOPATH/src/github.com/nightlyone/lockfile
$ go test

you should see output like this:
$ go test
PASS
ok  	github.com/nightlyone/lockfile	0.011s

@matjam , the test result of nightlyone/lockfile as follows:
/go/src/github.com/nightlyone/lockfile $ go test
Error locking lockfile:  link /go/src/github.com/nightlyone/lockfile/test_lockfile.pid.397504553 /go/src/github.com/nightlyone/lockfile/test_lockfile.pid: operation not permitted
--- FAIL: TestBasicLockUnlock (0.00s)
--- FAIL: TestRogueDeletion (0.00s)
        lockfile_test.go:129: link /go/src/github.com/nightlyone/lockfile/test_lockfile.pid.695546947 /go/src/github.com/nightlyone/lockfile/test_lockfile.pid: operation not permitted
--- FAIL: TestRogueDeletionDeadPid (0.00s)
        lockfile_test.go:158: link /go/src/github.com/nightlyone/lockfile/test_lockfile.pid.451082950 /go/src/github.com/nightlyone/lockfile/test_lockfile.pid: operation not permitted
--- FAIL: TestRemovesStaleLockOnDeadOwner (0.01s)
        lockfile_test.go:202: link /go/src/github.com/nightlyone/lockfile/test_lockfile.pid.451935720 /go/src/github.com/nightlyone/lockfile/test_lockfile.pid: operation not permitted
--- FAIL: TestInvalidPidLeadToReplacedLockfileAndSuccess (0.01s)
        lockfile_test.go:231: unexpected error: link /go/src/github.com/nightlyone/lockfile/test_lockfile.pid.088922970 /go/src/github.com/nightlyone/lockfile/test_lockfile.pid: operation not permitted
FAIL
exit status 1
FAIL    github.com/nightlyone/lockfile  0.043s

@jonahfang yeah something is wrong with that filesystem.
still need to see output of "mount"

I am being run inside a container,  there is not an existing sm.lock file. I use image registry.cn-hangzhou.aliyuncs.com/fangzx/govim_1.0.

But I use gdlide and godep well.

But when it's not in a container, it works fine. So...

I use glide and godep in the same docker container well.

That really means nothing, just because other things work, doesn't mean the container isn't broken, or the way you're using it isn't broken, or that there isn't something wrong with your kernel, or ...
That "operation not permitted" message is coming from the kernel. It's a response to a system call for doing file operations on /go/src/github.com/nightlyone/lockfile/test_lockfile.pid.397504553
if the kernel is saying "no" that's a problem with either the container you're using, or docker, or something else on your system.

My docker container use ubuntu:14.04.

It's because it's trying to do a hard link inside the container. That container, or your system, doesn't allow that, so it's failing.

@sdboyer I think we need to change the way we do locking, this is another issue that's being caused by the hard link in the lockfile package; there was the ReFS issue #913 and now this.
@jonahfang are you running Windows 10 with ReFS for the underlying filesystem by any chance?

No, I use mac only.

Well, the container, or something, is not allow hard links in the filesystem, so that's why it's failing.

@jonahfang Can you please grab this library and test it in your container. You should get the following output:
$ go get gopkg.in/check.v1
$ go get github.com/theckman/go-flock
$ cd $GOPATH/src/github.com/theckman/go-flock
$ go test
OK: 7 passed
PASS
ok  	github.com/theckman/go-flock	0.012s

@matjam go test passed!

@matjam Any progress of this issue?

I need someone with ReFS on Windows to test the same module before I can recommend to @sdboyer that we switch to go-flock.
You'll have to find a workaround in the meantime. Not running inside a the container that you're using, for example. Other containers seem to handle links just fine.

Was there a patch for this? I have this exact problem, running a docker container inside an ubuntu trusty vagrant box on a mac.
I can create files inside the container just fine, it just won't do it via go dep.

Same issue on Termux (Android Linux distro). It's running busybox, and I can "solve" this using proot:
proot --link2symlink dep init

Any chance we can use symlinks instead to avoid this workaround?

Update :
$ proot --link2symlink dep ensure -add github.com/src-d/go-git
all dirs lacked any go code

And nothing in the vendor folder... So, better but not a workaround yet.

Well, It looks like the linkat syscall  get the EPERM as the following:
linkat(AT_FDCWD, "/shared/gopath/pkg/dep/sm.lock.735551337", AT_FDCWD, "/shared/gopath/pkg/dep/sm.lock", 0) = -1 EPERM (Operation not permitted)

/shared (vboxsf) is mounted by virtualbox and my host machine is the Mac osx.
It's recommended to set gopath local file system:)

@cpapidas Would appreciate it if you could test this PR and confirm that it works for you.

@matjam I had tried your PR and I always get the same error.
$ dep init
waiting for lockfile /root/gopath/pkg/dep/sm.lock
waiting for lockfile /root/gopath/pkg/dep/sm.lock

also if I run the dep with "official" master version I get the following error
$ dep init
waiting for lockfile /root/gopath/pkg/dep/sm.lock: Lockfile created, but doesn't exist

@cpapidas can you give us information about what system you're running dep on?
Is this being run inside a container?

Yes, I run this inside a docker container.

Containers: 7
 Running: 0
 Paused: 0
 Stopped: 7
Images: 51
Server Version: 17.06.2-ce
Storage Driver: aufs
 Root Dir: /var/lib/docker/aufs
 Backing Filesystem: extfs
 Dirs: 67
 Dirperm1 Supported: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: bridge host macvlan null overlay
 Log: awslogs fluentd gcplogs gelf journald json-file logentries splunk syslog
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 6e23458c129b551d5c9871e5174f6b1b7f6d1170
runc version: 810190ceaa507aa2727d7ae6f4790c76ec150bd2
init version: 949e6fa
Security Options:
 apparmor
 seccomp
  Profile: default
Kernel Version: 4.4.0-77-generic
Operating System: Ubuntu 16.04.2 LTS
OSType: linux
Architecture: x86_64
CPUs: 2
Total Memory: 7.797GiB
Name: ubuntu-xenial
ID: BAX7:PGP6:LPUZ:7QVL:VGGP:YVLI:34R5:Y3SE:SSSC:BCKK:2DWS:ZDYM
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Experimental: false
Insecure Registries:
 <IP_ADDRESS>/8
Live Restore Enabled: false

WARNING: No swap limit support

Okay. So, apparently aufs doesn't do either fnctl style flocking or hard linking in a standard way.
The notes in the "incompatible" section are relevant, if not particularly clear: http://aufs.sourceforge.net/aufs.html
@sdboyer we will need to do something else. We can't use a global lock file like this as a way to arbitrate who gets to write changes.

Please note that the bug seems to only reproduce on non-Linux docker hosts. So I believe it is specific to vboxfs and not to aufs.

@nightlyone actually I can reproduce in Termux, which is not running Docker.

I'm facing the same problem with Mac OS (host) -> Vagrant (virtualbox) -> Ubuntu 16 (guest). No docker. The $GOPATH folder inside ubuntu is a virtual synced folder from the host OS.
Running dep on the same folder directly on the host OS works.

if you're running into this problem, a workaround should be to use the latest tip and set DEPNOLOCK=1 in the environment.
just, make very sure that you don't run two dep processes at once. it's reasonably likely to corrupt your GOPATH/pkg/dep/sources cache 💥 🤷♂

This fixes the problem in virtualbox+docker.
Thanks @ayang64 and @sdboyer.

FWIW ran into the same issue reported here while building github.com/sachaos/todoist on FreeBSD with zfs filesystem which is extremely common on FreeBSD. Might this be a stdlib bug?
I added my notes in case this helps future people landing on this error.
> uname -a
FreeBSD wintermute.skunkwerks.at 12.0-CURRENT FreeBSD 12.0-CURRENT #17 r325010+e50db1d5c304(master): Thu Oct 26 13:59:17 UTC 2017     root@wintermute:/usr/obj/usr/src/sys/GENERIC  amd64

> go version
go version go1.9.1 freebsd/amd64

> dep version
dep:
 version     : devel
 build date  :
 git hash    :
 go version  : go1.9.1
 go compiler : gc
 platform    : freebsd/amd64

> dep ensure -v
dep ensure
waiting for lockfile /repos/go/pkg/dep/sm.lock: Lockfile created, but doesn't exist
waiting for lockfile /repos/go/pkg/dep/sm.lock: Lockfile created, but doesn't exist
waiting for lockfile /repos/go/pkg/dep/sm.lock: Lockfile created, but doesn't exist
waiting for lockfile /repos/go/pkg/dep/sm.lock: Lockfile created, but doesn't exist
waiting for lockfile /repos/go/pkg/dep/sm.lock: Lockfile created, but doesn't exist
waiting for lockfile /repos/go/pkg/dep/sm.lock: Lockfile created, but doesn't exist
waiting for lockfile /repos/go/pkg/dep/sm.lock: Lockfile created, but doesn't exist

requested test repos
> go get github.com/nightlyone/lockfile
>  cd $GOPATH/src/github.com/nightlyone/lockfile
> go test
Error locking lockfile:  link /repos/go/src/github.com/nightlyone/lockfile/test_lockfile.pid.431102591 /repos/go/src/github.com/nightlyone/lockfile/test_lockfile.pid: operation not permitted
--- FAIL: TestBasicLockUnlock (0.00s)
--- FAIL: TestRogueDeletion (0.00s)
	lockfile_test.go:129: link /repos/go/src/github.com/nightlyone/lockfile/test_lockfile.pid.949387529 /repos/go/src/github.com/nightlyone/lockfile/test_lockfile.pid: operation not permitted
--- FAIL: TestRogueDeletionDeadPid (0.00s)
	lockfile_test.go:158: link /repos/go/src/github.com/nightlyone/lockfile/test_lockfile.pid.448157908 /repos/go/src/github.com/nightlyone/lockfile/test_lockfile.pid: operation not permitted
--- FAIL: TestRemovesStaleLockOnDeadOwner (0.00s)
	lockfile_test.go:202: link /repos/go/src/github.com/nightlyone/lockfile/test_lockfile.pid.959225638 /repos/go/src/github.com/nightlyone/lockfile/test_lockfile.pid: operation not permitted
--- FAIL: TestInvalidPidLeadToReplacedLockfileAndSuccess (0.00s)
	lockfile_test.go:231: unexpected error: link /repos/go/src/github.com/nightlyone/lockfile/test_lockfile.pid.332173640 /repos/go/src/github.com/nightlyone/lockfile/test_lockfile.pid: operation not permitted
FAIL
exit status 1
FAIL	github.com/nightlyone/lockfile	0.004s

>  go get gopkg.in/check.v1
> go get github.com/theckman/go-flock
> cd $GOPATH/src/github.com/theckman/go-flock
> go test
OK: 7 passed
PASS
ok  	github.com/theckman/go-flock	0.003s

results when using workaround:
> env DEPNOLOCK=1 dep ensure -v
Gopkg.lock was already in sync with imports and Gopkg.toml
(1/22) Wrote golang.org/x/sys@master
(2/22) Wrote golang.org/x/text@master
(3/22) Wrote<EMAIL_ADDRESS>(4/22) Wrote github.com/spf13/afero@master
(5/22) Wrote gopkg.in/yaml.v2@v2
(6/22) Wrote<EMAIL_ADDRESS>(7/22) Wrote<EMAIL_ADDRESS>(8/22) Wrote<EMAIL_ADDRESS>(9/22) Wrote github.com/hashicorp/hcl@master
(10/22) Wrote<EMAIL_ADDRESS>(11/22) Wrote<EMAIL_ADDRESS>(12/22) Wrote<EMAIL_ADDRESS>(13/22) Wrote github.com/mitchellh/mapstructure@master
(14/22) Wrote<EMAIL_ADDRESS>(15/22) Wrote<EMAIL_ADDRESS>(16/22) Wrote<EMAIL_ADDRESS>(17/22) Wrote<EMAIL_ADDRESS>(18/22) Wrote github.com/pkg/browser@master
(19/22) Wrote<EMAIL_ADDRESS>(20/22) Wrote<EMAIL_ADDRESS>(21/22) Wrote github.com/spf13/jwalterweatherman@master
(22/22) Wrote<EMAIL_ADDRESS>
\o/ thanks!

oi. yeah, another reason to swap out the lib we use to do this 😢

This issue is still occurring, has there been any progress at to change locking mechanism?
This is happening on linux, no container

@RichyHBM Did you try export DEPNOLOCK=1 in the shell before running go dep as a workaround?
That is what we have done until the mechanism changes.

Yes, that works as expected but it shouldnt be the fix

Lock mechanism still problem with docker(golang:1.10-alpine) on Windows10Pro(hyper-v enable).
Btw workaround is ok.

I get this on windows on an exFAT filesystem as well


[Fix] Upgraded the packages with the latest version

Added a dev Dependency as fallback, there was a
packages babel-preset-react-app that was directly importing @babel/plugin-proposal-private-property-in-object, without defining the package.json, but was but was present in node_module for some other purposes, it could cause break in future,
This was a bug as create-react-app is no longer being maintained, thus it was unlikely the bug was going to be fixed.

Date:  20 Aug 2024

Developer Name: Shubham Raj

Issue Ticket Number

#55

Update the package in the Repo to current latest version, ensuring there is no breaks, and all the dependencies are working fine. Tests are are working as intended.
Documentation Updated?

[ ] Yes
[x] No

Under Feature Flag

[ ] Yes
[x] No

Database Changes

[ ] Yes
[x] No

Dev server is also working fine

Breaking Changes

[x] Yes
[ ] No

Development Tested?

[x] Yes
[ ] No

Screenshots

Build Working

Test Coverage

Test Coverage

Dev Server
screenshare

Why does this PR add a component?

Why does this PR add a component?

App was breaking at the App.js while running tests, the component was imported but and no such component was found in index.js


Better test Module#ancestors cases

Sort of related to https://github.com/ruby/spec/pull/244
This might be a better way to test that although they are related

@eregon - Updated with lvars

Looks great, thanks!


Initial tool for trajectory comparison

https://github.com/user-attachments/assets/5047ae35-07eb-495d-b38c-dbb0a4d487bf


Composition Helper seems broken (macOS Sierra)

Details

Browser and browser version: Chrome 59
OS version: MacOS Sierra 10.12.6
xterm.js version: 2.9.0

Steps to reproduce
Open the xterm.js demo page, keep holding down the o key. The composition helper should pop up and show a list of characters to select - instead nothing is shown. Still, using the arrow keys seems to toggle through the (invisible) composition popup and displays the selected character at the cursor position. When confirming the selection with enter, nothing happens. The same behaviour happens in Firefox, although sometimes the character is inserted on enter.

Troubleshooting
Part of the problem is that the textarea is rendered off-screen (left: -9999em) initially.
We wait for the compositionstart event, then we move the textarea to the cursor position - but as it seems, the compositionstart event is fired after MacOS draws the composition popup and only once a character is selected in the popup, causing it to render somewhere offscreen.
@Tyriar @parisk do you experience the same problem? Does this also happen on other platforms?

I've suspected it's been broken for a while, haven't tested though. Things break when browsers update unfortunately 🙁

Mmmmh... I actually wanted to try myself on a PR for #265 - which made me come across this issue. Terminal.app does not show the composition dialog at all, probably in favour of printing sequential characters, and I wonder if we shouldn't do the same. However, I'm concerned breaking the IME input. Unfortunately I don't know how to test IME composition.
@Tyriar Do you know of a way to test IME input on the Mac?

It's a bit of an involved process, particularly if you aren't familiar with the input methods. I typically test Chinese, Japanese and Korean as they all have different input styles, and then you need to test them all in various browsers. To add them you need to add the input method to the OS, I would need to google how to do it on Mac.
I wouldn't expect #265 to be related to this though, I thought that one was to do with moving over to the input even for input tracking?

I also thought so but the input event only fires if the input value of the textbox has changed. That means we can't catch arrow keys or key combinations or F keys and so on with it. Additionally, at least on my Mac, holding down a key does not repeat characters in an input or textbox, it only print a single character, which means that the input event will only fire once. The only listener that fires multiple times while the key is down seems to be the keypress event.
On 4 Aug 2017, at 22:09, Daniel Imms <notifications<EMAIL_ADDRESS>wrote:
It's a bit of an involved process, particularly if you aren't familiar with the input methods. I typically test Chinese, Japanese and Korean as they all have different input styles, and then you need to test them all in various browsers. To add them you need to add the input method to the OS, I would need to google how to do it on Mac.
I wouldn't expect #265https://github.com/sourcelair/xterm.js/issues/265 to be related to this though, I thought that one was to do with moving over to the input even for input tracking?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://github.com/sourcelair/xterm.js/issues/836#issuecomment-320342078, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ACqCv4uH97irEFHSJxDnmThhwClQIZDLks5sU3qFgaJpZM4OtEkE.

I believe this is fixed

Yes. We ignore this kind of composition and repeat the original character instead - which is what Terminal.app is doing.


chore: Fixed client documentation deployment job

This PR fixes the deployment job of the documentation for the client

Hi there, thank you for the fix !


[New Feature]: 🎮Implementation of Tutorial and Help Section for Game Rules and Mechanics

🎮 feature Request
feature logic and basic description
Feature Logic:
The feature involves integrating a tutorial or help section into the game interface to provide players with clear instructions on game rules and mechanics.
Upon accessing the tutorial or help section, players will be presented with comprehensive information detailing the objective of the game, basic rules, gameplay mechanics, turn structure, scoring, and any special features or mechanics.
The tutorial or help section will guide players through setting up the game board, understanding how to play, and offer tips and strategies for optimal gameplay.
It will be easily accessible from the main menu or through in-game prompts to ensure that players have convenient access to assistance whenever needed.
Feature Brief:
The tutorial or help section serves as an invaluable resource for both new and experienced players, offering clarity on game mechanics and fostering a deeper understanding of gameplay strategies.
By providing detailed instructions and guidance, the feature enhances the overall user experience and reduces confusion or frustration among players.
Accessible at any time during gameplay, the tutorial or help section empowers players to quickly reference rules and mechanics, enabling smoother progression and increased engagement with the game.
With this feature, players can confidently navigate the game's complexities, leading to a more enjoyable and rewarding gaming experience overall.
Point down the features
feature points
Comprehensive Instructional Content: The feature will include a thorough explanation of game rules, mechanics, objectives, and any special features or mechanics present in the game.
Clear Visual Presentation: Information will be presented in a visually appealing and easy-to-understand format, utilizing text, images, and possibly animations to convey concepts effectively.
Step-by-Step Guidance: Players will be guided through each aspect of the game, from initial setup to advanced strategies, in a logical sequence, ensuring a smooth learning curve.
Select program in which you are contributing
GSSoC24
Code of Conduct

[X] I follow CONTRIBUTING GUIDELINE of this project.

Hello @Durgesh4993
Assign me this issue


In create/restore account, check if the name already exists

The gnokey command line to create an account checks if the key name already exists:
% ./build/gnokey add -recover jefft0
Override the existing name jefft0 [y/n]:

However, the example app Create and Restore account functions replace the existing account if it has the same name. The example app should check if an account with the same name exists and prompt the user, like the gnokey command line.
As in the gnokey command line, maybe we should add GetKeyInfoByName to the GnoMobile API.

Resolved by PR #63.


How to use Pivot Table on Mac ?

Hello,
I downloaded your pivot files but they are not working on Mac's safari browser as well as on chrome. Can you please help me out like how will I be able to run them on Mac.

Are you able to run the examples at https://pivottable.js.org/ on those
browsers? They work fine for me on Mac of course... maybe try copying them
and figuring out what you missed :)
On Tue, Oct 24, 2017 at 06:14 Siddhant Mahajani<EMAIL_ADDRESS>wrote:

Hello,
I downloaded your pivot files but they are not working on Mac's safari
browser as well as on chrome. Can you please help me out like how will I be
able to run them on Mac.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/nicolaskruchten/pivottable/issues/814, or mute the
thread
https://github.com/notifications/unsubscribe-auth/AAMbA6BYAhA-MLO8c2SsP4S3ykImTtgMks5svbihgaJpZM4QELPx
.


feat(vulnsrc/nvd): add CVSS v4.0

Description
Supports NVD CVSS V4.0 Schema with Trivy DB.
https://github.com/aquasecurity/vuln-list-update/pull/297
How has this been tested?
I've run the test cases using make test and all of them passed.

I could not do it since NVD only publishes CVSSv4 scored by other CNAs and has not yet published CVSSv4 by NVD itself

I saw it too. I renamed source field in few CVE files (to test) and it worked well.


Notebooks: pasting notebook JSON into notebook should automatically add cells

I've been sharing notebooks with others and keep running to the issue of easily adding/importing notebooks from others. Currently the steps that I do is:

Copy notebook JSON
Create new txt file
Paste notebook json and save
Rename output to {name}.github-issue
View notebook

I would LOVE it if I could just paste in the JSON data into any notebook an have it automatically add the cells to my notebook. Be it on an empty notebook or in an existing one.

We don't know yet if ppl will give it thumbups or not yet.

This sounds relevant to this where we need the functionality of automatically switching https://github.com/microsoft/vscode/issues/81832


Page-translator in Thunderbird

Page-translator is a good plugin that make the job.
I use it in Firefox and tried to install it also in Thunderbird.
I can successfully install the .xpi in Thunderbird (52.6.0 (64 bits) Linux) but the addon is unusable : no handling icon (because no url bar).
Maybe it could be +- easily ported to be compliant as Thunderbird extension ?
What do you think about that?

I think it's an interesting and useful idea. I don't use Thunderbird, but I welcome pull requests if someone is able to contribute.


[ci:component:github.com/gardener/terraformer:v1.5.0->v2.0.0]

*Release Notes:
Terraformer version `v2` now requires to pass in ConfigMap and Secret names via command line flags instead of environment variables and the base command of the docker image has changed to `/terraformer`. Please see the [examples](https://github.com/gardener/terraformer/tree/master/example) for more details.

If your provider extension is deploying terraformer Pods via gardener's [terraformer library](https://github.com/gardener/gardener/tree/master/extensions/pkg/terraformer), please make sure, that you use at least<EMAIL_ADDRESS>and set `terraformer.UseV2(true)` in order to deploy a PodSpec, that is compatible with terraformer `v2`.

Terraformer was rewritten in go and now watches the terraform state file in order to continuously update the state ConfigMap to not lose any relevant infrastructure state.

You can use `make start-dev-container` to start a docker container which can run terraformer and tests in an isolated environment for development and testing. Also `make start` will run terraformer commands in such a development container.

You can now run a small e2e test, that creates some lightweight resource on AWS via terraform, by executing `make test-e2e`.

/close
This requires some adaption (ref https://github.com/gardener/gardener-extension-provider-gcp/pull/190) and is planned to be done after provider-gcp has been successfully released and running on a productive gardener installation with terraformer v2.


the example model of 'SGPT-125M-weightedmean-nli-bitfit' not compatible with ST

hi, I tried to run the example be_st with 'Muennighoff/SGPT-125M-weightedmean-nli-bitfit'
but it throw out the exception as follows:
/usr/local/lib/python3.8/site-packages/sentence_transformers/models/Pooling.py:120 in load TypeError: __init__() got an unexpected keyword argument 'pooling_mode_weightedmean_tokens'
Is ’pooling_mode_weightedmean_tokens‘ in the config.json changed to ’pooling_mode_mean_tokens’ ？

Can you try what is explained here: https://github.com/Muennighoff/sgpt/issues/14#issuecomment-1352628910
You're likely using an old ST version

I got the same error with the computing of the web interface at https://huggingface.co/bigscience-data/sgpt-bloom-1b7-nli a moment ago.

I got the same error with the computing of the web interface at https://huggingface.co/bigscience-data/sgpt-bloom-1b7-nli a moment ago.

Yes that's because the web interface is not using the latest ST version


Implement TextHighlighter

Should we provide it in the core?

https://github.com/scrivo/highlight.php is still a good solution for it.

not sure if we really need support for it in the core... maybe better create it as an official extension?
In general I think we have a bit too many things in the core which solve only very specific use cases.
Imo we should use 2.1 to clean up in this regard a little bit.

I was thinking about highlighting of error pages / debug and apidoc rendered pages. Not sure it can be done w/o requiring more code in the core.

okay, makes sense.

as can be concluded from this discussion, it can be a whole architecture on its own. Out of the cores scope.


Parse nested schemas inside manifests

The way that you can tell these apart from nested references (and eventually nested entities) is that these are anonymous. So:

{foo: {bar: Text, far: Long}} is inline structure; foo.bar and foo.far will work but foo isn't a reference or sharable with other entities without copying.
{foo: &Thing {bar: Text, far: Long}}: foo is a sharable reference
{foo: Thing {bar: Text, far: Long}}: foo is stored normalized as a reference but the particle will treat it like it's inlined. foo.bar and foo.far will work, foo can be shared with other entities and you don't need to copy it.

{foo: {bar: Text, far: Long}} is inline structure; foo.bar and foo.far will work but foo isn't a reference or sharable with other entities without copying.

vs

{foo: Thing {bar: Text, far: Long}}

This seems kind of surprising to me, that simply giving a Schema a name makes it shareable vs not. It seems to violate the principle of least surprise. Maybe there's an actual elegant reason for it, but I think someone coming to Arcs for the first time would be like 'whaaa?' because it's not obvious.

I've modified this to use an inline keyword instead:
{foo: inline Thing {bar: Text, far: Long}}
PTAnotherL


[Request for Experiment] Sysroot building functionality

Summary
This RFE proposes adding Xargo's functionality of building a sysroot on the fly to Cargo as a nightly-only feature.
Motivation
Many (all?) no-std / embedded developers have to Install One More Tool because the ability to build core / std from source is missing in Cargo. Apart from the inconvenience, the Xargo wrapper is from for perfect: it can trigger unnecessary sysroot rebuilds because it doesn't replicate Cargo fingerprinting mechanism; it can sometimes fail to trigger a necessary sysroot rebuild (japaric/xargo#189); it doesn't track changes in the sysroot source code (japaric/xargo#139); and it doesn't understand the +nightly command line argument because it's not a rustup shim (japaric/xargo#123) among other deviations from the behavior users expect from a built-in Cargo subcommand.
Apart from all the issues Xargo also ties no-std / embedded development to the nightly channel. This experiment will hopefully be a first step towards enabling no-std / embedded development on the stable channel.
Implementation details
User interface: [sysroot]
A [sysroot] section will be added to Cargo configuration file:
# .cargo/config
[sysroot]
rust-src = "~/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src"

[sysroot.core]
path = "libcore" # paths are relative to `rust-src` when not absolute
stage = 1 # see multi-stage builds section below

[sysroot.compiler_builtins]
git = "https://github.com/rust-lang-nursery/compiler-builtins"
features = ["mem"]
stage = 2

There the user can specify the crates that will be included in the sysroot. Only path and git dependencies are allowed. The [sysroot.rust-src] setting is a convenience that lets the user use relative paths.
Behavior
If a [sysroot] setting is found in Cargo configuration file:
Cargo will (re)build the sysroot crates for the target and place the build artifacts in $TARGET_DIR/sysroot/lib/rustlib/$TARGET before executing subcommands that involve invoking rustc or rustdoc. Then when invoking the actual subcommand Cargo will append the argument --sysroot=$TARGET_DIR/sysroot to all its rustc and rustdoc invocations except the ones used to build build scripts (build.rs).
The usual fingerprinting mechanism applies to the sysroot build: for example changes to [profile] in Cargo.toml and changes in the sysroot source code will trigger a sysroot rebuild.
The sysroot crates will always be built using the release profile to not regress the performance and binary size of dev builds when switching to builds that use [sysroot].
Multi-stage builds
There are crates in the std facade, like the test crate, that have implicit dependencies on other members of the facade. Building sysroots that include these crates require multi-stage builds.
In multi-stage builds the sysroot will be build as follows: all the crates in the first stage are build against the default sysroot; then all the crates in the second stage are build using the stage 1 artifacts as a custom sysroot (i.e. --sysroot is passed to rustc); then all the crates in the third stage are build using the stage 1 and 2 artifacts as a custom sysroot; the process continues until all stages are built; finally the artifacts of all the stages are placed in $TARGET_DIR/sysroot/lib/rustlib/$TARGET.
Potential additions / tweaks
Xargo doesn't require the user to specify a rust-src setting because it assumes that both rustup and the rust-src component are installed and it uses rustc --print sysroot to get the path to the Rust source. We could do the same here by either: (a) probing for the existence of rust-src and printing a helpful error message when it's not installed, or (b) committing to always ship rust-src with the toolchain.
Xargo always rebuilds the sysroot in release mode but we could have the Cargo implementation use the dev profile when --release is not passed to the subcommand.
Future steps
Revisit rust-lang/rfcs#1133 to see if there's a desire for the changes proposed there that aren't included in this RFE: eliminating the concept of the sysroot, versioning the crates in the std facade, etc.
UPDATE(2018-01-20): Don't pass --sysroot to rustc when building build scripts (build.rs). Those should be build against the default sysroot because they always run on the host. If the build scripts were to be build against the custom sysroot the sysroot would need to contain the rust-std artifacts of the host and that would require copying (or linking) those into $TARGET_DIR/sysroot/lib/rustlib/$HOST, which is a waste of space.

cc @alexcrichton @nrc @Ericson2314

I think it would be good to restrict the crates which can be placed in the sysroot to the usual ones we would expect there, unless there is good reason not to?
Are there other parts of the program other than build scripts that should be built with the host profile? Proc macros/custom derives seem like one thing. Are there more?
I wonder if there is a more user-friendly surface syntax we could layer on top of the custom sysroot facility? It seems like for any given platform and version, the sysroot setup will be the same, so it would be good to be able to set that all up once and then point to it from user crates, rather than having to spell it out each time.

rust-src = "~/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src"

Could this path be discovered by running rustc --print sysroot, rather than being hard-coded? It’d be nice to be able to switch between cargo +nightly build and e.g. cargo +nightly-2018-01-24 build, or different nightly dates in different projects, without editing a config file.

That's what's proposed in the potential additions/tweaks section

@jdub

The tweak to assume the presence and location of the rust-src component is worth adopting. It makes Xargo so easy to use, and I suspect* it's what most users want to build. Using your own sysroot sources is preeeeeeetty advanced.

For my usecase (wasm) I want to be able to get a copy of the rust std, hack around with it to create my own integrations with JS directly in libstd, and then build it. I'd say that experimenting with libstd (either to experiment with targets, or with the intent of making PRs against the main Rust repo) should not be an advanced use case and any solution should take care to have it as a first-class feature.
That's not to say that we can't make the rust-src assumption! I just want to make sure that in the process we don't throw away the plan to have correct rebuilds of sysroots when files are changed (because we assume rust-src components are immutable) - this is a nasty pain point of xargo today, as the RFE identifies.

Another thing to think about, if we integrate rustup functionality into Cargo, does this sysroot stuff interact with any of the cross-compilation-oriented toolchain features from rustup?

I didn’t mean that Cargo would necessarily have rustup-specific functionality. Rustup could for example set an environment variable that Cargo (and rust-gdb!) would read to know where to find std sources. Or maybe have a conventional path relative to the rustc binary. (Or both.)

It's kind of the plan that we'll merge rustup and Cargo this year

I brought it up in https://github.com/rust-lang/rfcs/pull/1133#issuecomment-362354174 and @matklad posted a response https://github.com/rust-lang/rfcs/pull/1133#issuecomment-362355002, but for posterity I think it's better I mirror and elaborate my concerns on this proposal here.

The sysroot impedes high priority tools/tool features like incremental compilation and IDE support. Both those involve to storing more meta data and intermediate artifacts for per crate for caching purposes. It's far more easy to implement that if libraries are built the same way. Because then just one build system needs each create, and it's easier to ensure the adjustment will have the desired uniform effect on each crate.

Explicit boostrapping stages is...

...cumbersome. Cargo works by looking at the needed dependencies of each create, and then deriving a plan automatically. This roughly requires the user to hand-compute a topological sort, essentially doing Cargo's work.
...brittle. Small changes to the dependency graph can greatly change the topological sort, requiring the user to recompute everything all over again.
...error-prone. If the user writes the boostrapping stages wrong, only rustc, not Cargo will catch the mistake. Also, in the Rust code itself, each library may freely link all the libraries in the previous stage which may not be the author's intention.
....unperformant. A full dependency graph exposes more potential parallelism.

@Ericson2314, here's one more argument for, eventually, getting rid of sysroot altogether: currently, we have to use one set of compiler flags for stdlib, for all use cases. Building your own std should allow one to use the precise flags one wants. The particular problems today are:

stdlib today is compiled as release with debuginfo, and symbols blow up release binaries on linux tremendously. There are about 4MB of debuginfo in std, and all of this goes into hello-world =/
debug_assert is useless in std
you can't target-cpu=native stdlib

How are the stability guarantees around this? xargo breaks every now and then because something in rustc changes. (Though I assume if rustc CI tests that this works, that would happen less often.)
Also, compiling libstd requires nightly features -- so i we want to permit custom sysroots on stable, and people start patching libstd, they could easily end up in a situation where upgrading the compiler breaks their code.

Sounds like a great idea.

Looks by using https://github.com/rust-lang/wg-cargo-std-aware/issues/51 can resolve this issue


Unable to Access Azure Key Vault from Azure Batch Nodes Despite Necessary Permissions.

We are experiencing issues accessing Azure Key Vault secrets from a Python script stored in Azure Data Lake Storage. These scripts run on Azure Batch nodes with Linux OS. Despite granting the necessary permissions to the Batch nodes and configuring the identity as a system-assigned managed identity, we cannot access the Key Vault secrets. The pool allocation mode is set to BatchService.
We are using DefaultAzureCredential() to obtain credentials and access the secrets. However, we encounter the following error:
Error:
DefaultAzureCredential failed to retrieve a token from the included credentials.
Attempted credentials:
EnvironmentCredential: EnvironmentCredential authentication unavailable. Environment variables are not fully configured.
Visit https://aka.ms/azsdk/python/identity/environmentcredential/troubleshoot to troubleshoot this issue.
ManagedIdentityCredential: ManagedIdentityCredential authentication unavailable. The requested identity has not been assigned to this resource. Error: Unexpected response "{'error': 'invalid_request', 'error_description': 'Identity not found'}"
SharedTokenCacheCredential: SharedTokenCacheCredential authentication unavailable. No accounts were found in the cache.
AzureCliCredential: Azure CLI not found on path
AzurePowerShellCredential: PowerShell is not installed
AzureDeveloperCliCredential: Azure Developer CLI could not be found. Please visit https://aka.ms/azure-dev for installation instructions and then,once installed, authenticate to your Azure account using 'azd auth login'.
To mitigate this issue, please refer to the troubleshooting guidelines here at https://aka.ms/azsdk/python/identity/defaultazurecredential/troubleshoot.
Traceback (most recent call last):
File "test_variable_pipeline.py", line 27, in
retrieved_secret = client.get_secret(secret_name)
File "/usr/local/lib/python3.8/dist-packages/azure/core/tracing/decorator.py", line 78, in wrapper_use_tracer
return func(*args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/azure/keyvault/secrets/_client.py", line 64, in get_secret
bundle = self._client.get_secret(
File "/usr/local/lib/python3.8/dist-packages/azure/core/tracing/decorator.py", line 78, in wrapper_use_tracer
return func(*args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/azure/keyvault/secrets/_generated/_operations/_operations.py", line 751, in get_secret
pipeline_response: PipelineResponse = self._client._pipeline.run(  # pylint: disable=protected-access
File "/usr/local/lib/python3.8/dist-packages/azure/core/pipeline/_base.py", line 230, in run
return first_node.send(pipeline_request)
File "/usr/local/lib/python3.8/dist-packages/azure/core/pipeline/_base.py", line 86, in send
response = self.next.send(request)
File "/usr/local/lib/python3.8/dist-packages/azure/core/pipeline/_base.py", line 86, in send
response = self.next.send(request)
File "/usr/local/lib/python3.8/dist-packages/azure/core/pipeline/_base.py", line 86, in send
response = self.next.send(request)
[Previous line repeated 2 more times]
File "/usr/local/lib/python3.8/dist-packages/azure/core/pipeline/policies/_redirect.py", line 197, in send
response = self.next.send(request)
File "/usr/local/lib/python3.8/dist-packages/azure/core/pipeline/policies/_retry.py", line 531, in send
response = self.next.send(request)
File "/usr/local/lib/python3.8/dist-packages/azure/core/pipeline/policies/_authentication.py", line 135, in send
request_authorized = self.on_challenge(request, response)
File "/usr/local/lib/python3.8/dist-packages/azure/keyvault/secrets/_shared/challenge_auth_policy.py", line 137, in on_challenge
self.authorize_request(request, scope, tenant_id=challenge.tenant_id)
File "/usr/local/lib/python3.8/dist-packages/azure/core/pipeline/policies/_authentication.py", line 113, in authorize_request
self._token = self._credential.get_token(*scopes, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/azure/identity/_credentials/default.py", line 225, in get_token
token = super().get_token(*scopes, claims=claims, tenant_id=tenant_id, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/azure/identity/_credentials/chained.py", line 124, in get_token
raise ClientAuthenticationError(message=message)
azure.core.exceptions.ClientAuthenticationError: DefaultAzureCredential failed to retrieve a token from the included credentials.
Attempted credentials:
EnvironmentCredential: EnvironmentCredential authentication unavailable. Environment variables are not fully configured.
Visit https://aka.ms/azsdk/python/identity/environmentcredential/troubleshoot to troubleshoot this issue.
ManagedIdentityCredential: ManagedIdentityCredential authentication unavailable. The requested identity has not been assigned to this resource. Error: Unexpected response "{'error': 'invalid_request', 'error_description': 'Identity not found'}"
SharedTokenCacheCredential: SharedTokenCacheCredential authentication unavailable. No accounts were found in the cache.
AzureCliCredential: Azure CLI not found on path
AzurePowerShellCredential: PowerShell is not installed
AzureDeveloperCliCredential: Azure Developer CLI could not be found. Please visit https://aka.ms/azure-dev for installation instructions and then,once installed, authenticate to your Azure account using 'azd auth login'.
To mitigate this issue, please refer to the troubleshooting guidelines here at https://aka.ms/azsdk/python/identity/defaultazurecredential/troubleshoot.
We have connected to a Batch node and executed the following lines of code:
from azure.identity import DefaultAzureCredential
from azure.keyvault.secrets import SecretClient
credential = DefaultAzureCredential()
key_vault_url = "https://.vault.azure.net/"
client = SecretClient(vault_url=key_vault_url, credential=credential)
secret_name = ""
retrieved_secret = client.get_secret(secret_name)
print(f"Secret: {retrieved_secret.value}")
We are unsure where the issue lies and are looking for guidance on resolving this problem or alternative methods to achieve the same result. Any assistance would be greatly appreciated.

Thanks for reaching out.
Please follow https://learn.microsoft.com/en-us/azure/batch/managed-identity-pools to configure and use managed identity in batch pools


Multi Counter per Model

Hi,
Is it possible to have two counter per model?
Let's say there is App\Post model, counter one is called views and the another counter is called clicks.
Please let me know if this possible.

no sorry I made it for visits . but we can add this feature easily later

done !


WIP: move human filter for non-human hosts after subsampling step

For performance reasons.

LGTM. What more is required for moving a pipeline step?

This should be all... just left "WIP" because I haven't tested it yet.


Update types to better match reality.

This also fixes an issue I discovered while figuring out what type indentExclusionsRanges was supposed to be. MagicString#clone does not appear to work when the object was constructed with a non-tuple. It seemed to expect an array of nodes – or something with start and end properties – but generated an array of tuples. Calling #clone multiple times no matter what the starting value would have resulted in an incorrect clone. I'm not 100% sure that I picked the right type, but this seemed the most consistent choice.

Excellent, thank you! Released 0.21.3.


Postgres: Add support for partitioned tables

Postgres natively supports partitioned tables
https://www.postgresql.org/docs/current/ddl-partitioning.html

How large of a lift would it be to support partitioned tables? Is that something I could contribute to? I'm guessing some involved logic in the create stage, given you have to declare the partitions manually in PostgreSQL.
I have an incremental_by_time model with 1.7B rows, and maintaining the (necessary) indexes has caused things to slow dramatically. Partitioning by month, so at least the insert by day only has to reference one month's worth of data, would likely help tremendously.
@izeigerman

It would be a new concept for SQLMesh, right now its concept of partitioning is "tell the underlying engine what columns to partition on and let it handle it transparently".
This would be active manual partitioning where SQLMesh itself has to create, drop and keep track of partitions. On the plus side, adding this for Postges would also benefit Hive/Athena which has the same problem as Postgres

Postgres supports  Declarative Partitioning, as described in the link above, 5.11.2. Declarative Partitioning.

Indeed it does, and even with Declarative Partitioning you still have to manually create the partitions (you just don't have to manually attach and detach them).
For example:
create table foo (id int, day date) partition by range (day);

insert into foo (id, day) values (1, '2023-01-01'::date);
--FAIL: ERROR: no partition of relation "foo" found for row.  Detail: Partition key of the failing row contains (day) = (2023-01-01).

create table foo_2023 partition of foo for values from ('2023-01-01') to ('2024-01-01');

insert into foo (id, day) values (1, '2023-01-01'::date); --succeeds

insert into foo (id, day) values (1, '2024-01-01'::date);
--fails with ERROR: no partition of relation "foo" found for row  Detail: Partition key of the failing row contains (day) = (2024-01-01).

Today, SQLMesh assumes that the DB will automatically create partitions when you start trying to insert data, which isnt the case for Postrgres. So we would have to track partitions manually and issue the correct commands to create + drop them so that we can successfully insert data into the table.

Would it simplify things to tie back to snapshot intervals? So forcing partitions to align (but contain possibly multiple of) with the interval unit?
Given the way SQLMesh also creates all the physical tables, then evaluates models, making it inherit from the parent table with pg_partman would be possible, right?
Then as snapshots are evaluated, it first checks if the physical table has a partition available for the interval.
I’m not sure if incremental by time and range based partitioning have to go together in SQLMesh, but to me, it seems simpler to implement if both are used in conjunction.

Yes, for incremental models partitioned by time, the plan is to use RANGE-based partitioning. I'm debating whether to make it multiples of the interval unit specifically or copy what Iceberg did which is nice and simple (make year(time_col), month(time_col), day(time_col) or hour(time_col) the available options).
I'm also looking at how we can add LIST and HASH based partitioning because this lines up with partitioning strategies of other engines (identity and bucket in Iceberg).
I'm not planning to add a dependency on pg_partman at this stage. The SQLMesh Postgres adapter will be responsible for ensuring partitions exist on the target table prior to inserting data and should work out of the box on any Postgres instance without special configuration.


script.js:192 Uncaught (in promise) ReferenceError: pdfjsLib is not defined

when upload a PDF file， i got this problem:  script.js:192 Uncaught (in promise) ReferenceError: pdfjsLib is not defined

me too

Hi @lkunxyz @nickboyabc sorry to hear you're facing these issues. I have made major updates since and launched a new web app on www.dara.chat, you should not be facing any of these errors there. Please let me know if you still do!


Diff dot ns frames

issue #279

why do the ci tests fail now?

in this branch I made minor changes to dot output (peers node name ) in both connlist and diff.
I need to run test-update for the new dot output that was added by the "connlist doc update branch" to get the fixed ones for this branch.


Very slow installation

While attempting to install this package with pip at NERSC, it seems like pip / poetry is downloading many different versions of dependency wheels, amounting to hundreds of MB.  Is this the expected behavior?  Output of
python3 -m pip install --prefix <foo> .

was the following:
Processing /global/cscratch1/sd/sobs/build/mapsims-2.4.0
  DEPRECATION: A future pip version will change local packages to be built in-place without first copying to a temporary directory. We recommend you use --use-feature=in-tree-build to test your packages with this new behavior before it becomes the default.
   pip 21.3 will remove support for this functionality. You can find discussion regarding this at https://github.com/pypa/pip/issues/7555.
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
    Preparing wheel metadata: started
    Preparing wheel metadata: finished with status 'done'
Requirement already satisfied: healpy<2.0.0,>=1.14.0 in /global/common/software/cmb/cori/cmbenv-gcc_20210527/cmbenv_aux/lib/python3.7/site-packages (from mapsims==2.4.0) (1.14.0)
Requirement already satisfied: so_noise_models==3.1.1 in /global/common/software/sobs/cori/gcc_20210629/lib/python3.7/site-packages (from mapsims==2.4.0) (3.1.1)
Requirement already satisfied: numpy<2.0.0,>=1.18.0 in /global/common/software/cmb/cori/cmbenv-gcc_20210527/cmbenv_python/lib/python3.7/site-packages (from mapsims==2.4.0) (1.19.4)
Requirement already satisfied: pysm3<4.0.0,>=3.3.0 in /global/common/software/cmb/cori/cmbenv-gcc_20210527/cmbenv_aux/lib/python3.7/site-packages (from mapsims==2.4.0) (3.3.0)
Requirement already satisfied: astropy<5,>=4 in /global/common/software/cmb/cori/cmbenv-gcc_20210527/cmbenv_aux/lib/python3.7/site-packages (from mapsims==2.4.0) (4.2.1)
Requirement already satisfied: toml<0.11.0,>=0.10.1 in /global/common/software/cmb/cori/cmbenv-gcc_20210527/cmbenv_python/lib/python3.7/site-packages (from mapsims==2.4.0) (0.10.2)
Requirement already satisfied: pyyaml<6,>=5 in /global/common/software/cmb/cori/cmbenv-gcc_20210527/cmbenv_python/lib/python3.7/site-packages (from mapsims==2.4.0) (5.3.1)
Requirement already satisfied: so-pysm-models<3.0.0,>=2.4.0 in /global/common/software/sobs/cori/gcc_20210629/lib/python3.7/site-packages (from mapsims==2.4.0) (2.4.0)
Collecting pixell<0.11.0,>=0.10.3
  Downloading pixell-0.10.3-cp37-cp37m-manylinux2010_x86_64.whl (9.8 MB)
Collecting importlib-metadata<4.0.0,>=3.4.0
  Downloading importlib_metadata-3.10.1-py3-none-any.whl (14 kB)
Requirement already satisfied: pyerfa in /global/common/software/cmb/cori/cmbenv-gcc_20210527/cmbenv_aux/lib/python3.7/site-packages (from astropy<5,>=4->mapsims==2.4.0) (2.0.0)
Requirement already satisfied: scipy in /global/common/software/cmb/cori/cmbenv-gcc_20210527/cmbenv_python/lib/python3.7/site-packages (from healpy<2.0.0,>=1.14.0->mapsims==2.4.0) (1.5.3)
Requirement already satisfied: six in /global/common/software/cmb/cori/cmbenv-gcc_20210527/cmbenv_python/lib/python3.7/site-packages (from healpy<2.0.0,>=1.14.0->mapsims==2.4.0) (1.16.0)
Requirement already satisfied: matplotlib in /global/common/software/cmb/cori/cmbenv-gcc_20210527/cmbenv_python/lib/python3.7/site-packages (from healpy<2.0.0,>=1.14.0->mapsims==2.4.0) (3.2.2)
Requirement already satisfied: zipp>=0.5 in /global/common/software/cmb/cori/cmbenv-gcc_20210527/cmbenv_python/lib/python3.7/site-packages (from importlib-metadata<4.0.0,>=3.4.0->mapsims==2.4.0) (3.4.1)
Requirement already satisfied: typing-extensions>=3.6.4 in /global/common/software/cmb/cori/cmbenv-gcc_20210527/cmbenv_python/lib/python3.7/site-packages (from importlib-metadata<4.0.0,>=3.4.0->mapsims==2.4.0) (<IP_ADDRESS>)
Requirement already satisfied: setuptools>=39 in /global/common/software/cmb/cori/cmbenv-gcc_20210527/cmbenv_python/lib/python3.7/site-packages (from pixell<0.11.0,>=0.10.3->mapsims==2.4.0) (49.6.0.post20210108)
Requirement already satisfied: cython>=0.28 in /global/common/software/cmb/cori/cmbenv-gcc_20210527/cmbenv_python/lib/python3.7/site-packages (from pixell<0.11.0,>=0.10.3->mapsims==2.4.0) (0.29.23)
Requirement already satisfied: pytest>=4.6 in /global/common/software/cmb/cori/cmbenv-gcc_20210527/cmbenv_python/lib/python3.7/site-packages (from pixell<0.11.0,>=0.10.3->mapsims==2.4.0) (6.2.4)
Requirement already satisfied: h5py>=2.7<=2.10 in /global/common/software/cmb/cori/cmbenv-gcc_20210527/cmbenv_aux/lib/python3.7/site-packages (from pixell<0.11.0,>=0.10.3->mapsims==2.4.0) (3.2.1)
Requirement already satisfied: python-dateutil>=2.7 in /global/common/software/cmb/cori/cmbenv-gcc_20210527/cmbenv_python/lib/python3.7/site-packages (from pixell<0.11.0,>=0.10.3->mapsims==2.4.0) (2.8.1)
Requirement already satisfied: pytest-cov>=2.6 in /global/common/software/cmb/cori/cmbenv-gcc_20210527/cmbenv_aux/lib/python3.7/site-packages (from pixell<0.11.0,>=0.10.3->mapsims==2.4.0) (2.12.0)
Requirement already satisfied: Pillow>=5.3.0 in /global/common/software/cmb/cori/cmbenv-gcc_20210527/cmbenv_python/lib/python3.7/site-packages (from pixell<0.11.0,>=0.10.3->mapsims==2.4.0) (7.2.0)
Requirement already satisfied: coveralls>=1.5 in /global/common/software/cmb/cori/cmbenv-gcc_20210527/cmbenv_aux/lib/python3.7/site-packages (from pixell<0.11.0,>=0.10.3->mapsims==2.4.0) (3.1.0)
Requirement already satisfied: requests>=1.0.0 in /global/common/software/cmb/cori/cmbenv-gcc_20210527/cmbenv_python/lib/python3.7/site-packages (from coveralls>=1.5->pixell<0.11.0,>=0.10.3->mapsims==2.4.0) (2.24.0)
Requirement already satisfied: coverage<6.0,>=4.1 in /global/common/software/cmb/cori/cmbenv-gcc_20210527/cmbenv_aux/lib/python3.7/site-packages (from coveralls>=1.5->pixell<0.11.0,>=0.10.3->mapsims==2.4.0) (5.5)
Requirement already satisfied: docopt>=0.6.1 in /global/common/software/cmb/cori/cmbenv-gcc_20210527/cmbenv_aux/lib/python3.7/site-packages (from coveralls>=1.5->pixell<0.11.0,>=0.10.3->mapsims==2.4.0) (0.6.2)
Requirement already satisfied: cached-property in /global/common/software/cmb/cori/cmbenv-gcc_20210527/cmbenv_aux/lib/python3.7/site-packages (from h5py>=2.7<=2.10->pixell<0.11.0,>=0.10.3->mapsims==2.4.0) (1.5.2)
Requirement already satisfied: cycler>=0.10 in /global/common/software/cmb/cori/cmbenv-gcc_20210527/cmbenv_python/lib/python3.7/site-packages (from matplotlib->healpy<2.0.0,>=1.14.0->mapsims==2.4.0) (0.10.0)
Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /global/common/software/cmb/cori/cmbenv-gcc_20210527/cmbenv_python/lib/python3.7/site-packages (from matplotlib->healpy<2.0.0,>=1.14.0->mapsims==2.4.0) (2.4.7)
Requirement already satisfied: kiwisolver>=1.0.1 in /global/common/software/cmb/cori/cmbenv-gcc_20210527/cmbenv_python/lib/python3.7/site-packages (from matplotlib->healpy<2.0.0,>=1.14.0->mapsims==2.4.0) (1.3.1)
Requirement already satisfied: numba>0.49.1 in /global/common/software/cmb/cori/cmbenv-gcc_20210527/cmbenv_python/lib/python3.7/site-packages (from pysm3<4.0.0,>=3.3.0->mapsims==2.4.0) (0.53.1)
INFO: pip is looking at multiple versions of pyparsing to determine which version is compatible with other requirements. This could take a while.
Collecting pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1
  Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
  Downloading pyparsing-2.4.6-py2.py3-none-any.whl (67 kB)
  Downloading pyparsing-2.4.5-py2.py3-none-any.whl (67 kB)
  Downloading pyparsing-2.4.4-py2.py3-none-any.whl (67 kB)
  Downloading pyparsing-2.4.3-py2.py3-none-any.whl (67 kB)
  Downloading pyparsing-2.4.2-py2.py3-none-any.whl (65 kB)
  Downloading pyparsing-<IP_ADDRESS>-py2.py3-none-any.whl (64 kB)
INFO: pip is looking at multiple versions of pyparsing to determine which version is compatible with other requirements. This could take a while.
  Downloading pyparsing-2.4.0-py2.py3-none-any.whl (62 kB)
  Downloading pyparsing-2.3.1-py2.py3-none-any.whl (61 kB)
  Downloading pyparsing-2.3.0-py2.py3-none-any.whl (59 kB)
  Downloading pyparsing-2.2.2-py2.py3-none-any.whl (57 kB)
  Downloading pyparsing-2.2.1-py2.py3-none-any.whl (57 kB)
INFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. If you want to abort this run, you can press Ctrl + C to do so. To improve how pip performs, tell us what happened here: https://pip.pypa.io/surveys/backtracking
  Downloading pyparsing-2.2.0-py2.py3-none-any.whl (56 kB)
  Downloading pyparsing-2.1.10-py2.py3-none-any.whl (56 kB)
  Downloading pyparsing-2.1.9-py2.py3-none-any.whl (55 kB)
  Downloading pyparsing-2.1.8-py2.py3-none-any.whl (54 kB)
  Downloading pyparsing-2.1.7-py2.py3-none-any.whl (53 kB)
  Downloading pyparsing-2.1.5-py2.py3-none-any.whl (42 kB)
  Downloading pyparsing-2.1.4-py2.py3-none-any.whl (40 kB)
  Downloading pyparsing-2.1.3-py2.py3-none-any.whl (39 kB)
  Downloading pyparsing-2.1.1-py2.py3-none-any.whl (38 kB)
  Downloading pyparsing-2.1.0-py2.py3-none-any.whl (38 kB)
  Downloading pyparsing-2.0.7-py2.py3-none-any.whl (38 kB)
  Downloading pyparsing-2.0.6-py2.py3-none-any.whl (38 kB)
  Downloading pyparsing-2.0.5-py2.py3-none-any.whl (38 kB)
  Downloading pyparsing-2.0.3-py2.py3-none-any.whl (37 kB)
  Downloading pyparsing-2.0.2.zip (1.3 MB)
  Downloading pyparsing-2.0.1.zip (1.3 MB)
INFO: pip is looking at multiple versions of pillow to determine which version is compatible with other requirements. This could take a while.
Collecting Pillow>=5.3.0
  Downloading Pillow-8.2.0-cp37-cp37m-manylinux1_x86_64.whl (3.0 MB)
  Downloading Pillow-8.1.2-cp37-cp37m-manylinux1_x86_64.whl (2.2 MB)
  Downloading Pillow-8.1.1-cp37-cp37m-manylinux1_x86_64.whl (2.2 MB)
  Downloading Pillow-8.1.0-cp37-cp37m-manylinux1_x86_64.whl (2.2 MB)
  Downloading Pillow-8.0.1-cp37-cp37m-manylinux1_x86_64.whl (2.2 MB)
  Downloading Pillow-8.0.0-cp37-cp37m-manylinux1_x86_64.whl (2.2 MB)
  Downloading Pillow-7.2.0-cp37-cp37m-manylinux1_x86_64.whl (2.2 MB)
INFO: pip is looking at multiple versions of pillow to determine which version is compatible with other requirements. This could take a while.
  Using cached Pillow-7.1.2-cp37-cp37m-manylinux1_x86_64.whl (2.1 MB)
  Downloading Pillow-7.1.1-cp37-cp37m-manylinux1_x86_64.whl (2.1 MB)
  Downloading Pillow-7.1.0-cp37-cp37m-manylinux1_x86_64.whl (2.1 MB)
  Downloading Pillow-7.0.0-cp37-cp37m-manylinux1_x86_64.whl (2.1 MB)
  Downloading Pillow-6.2.2-cp37-cp37m-manylinux1_x86_64.whl (2.1 MB)
INFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. If you want to abort this run, you can press Ctrl + C to do so. To improve how pip performs, tell us what happened here: https://pip.pypa.io/surveys/backtracking
  Downloading Pillow-6.2.1-cp37-cp37m-manylinux1_x86_64.whl (2.1 MB)
  Downloading Pillow-6.2.0-cp37-cp37m-manylinux1_x86_64.whl (2.1 MB)
  Downloading Pillow-6.1.0-cp37-cp37m-manylinux1_x86_64.whl (2.1 MB)
  Downloading Pillow-6.0.0-cp37-cp37m-manylinux1_x86_64.whl (2.0 MB)
  Downloading Pillow-5.4.1-cp37-cp37m-manylinux1_x86_64.whl (2.0 MB)
  Downloading Pillow-5.4.0-cp37-cp37m-manylinux1_x86_64.whl (2.0 MB)
  Downloading Pillow-5.3.0-cp37-cp37m-manylinux1_x86_64.whl (2.0 MB)
INFO: pip is looking at multiple versions of kiwisolver to determine which version is compatible with other requirements. This could take a while.
Collecting kiwisolver>=1.0.1
  Downloading kiwisolver-1.3.1-cp37-cp37m-manylinux1_x86_64.whl (1.1 MB)
  Downloading kiwisolver-1.2.0-cp37-cp37m-manylinux1_x86_64.whl (88 kB)
  Downloading kiwisolver-1.1.0-cp37-cp37m-manylinux1_x86_64.whl (90 kB)
  Downloading kiwisolver-1.0.1-cp37-cp37m-manylinux1_x86_64.whl (89 kB)
INFO: pip is looking at multiple versions of cycler to determine which version is compatible with other requirements. This could take a while.
Collecting cycler>=0.10
  Downloading cycler-0.10.0-py2.py3-none-any.whl (6.5 kB)
INFO: pip is looking at multiple versions of kiwisolver to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of matplotlib to determine which version is compatible with other requirements. This could take a while.
Collecting matplotlib
  Downloading matplotlib-3.4.2-cp37-cp37m-manylinux1_x86_64.whl (10.3 MB)
INFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. If you want to abort this run, you can press Ctrl + C to do so. To improve how pip performs, tell us what happened here: https://pip.pypa.io/surveys/backtracking
  Downloading matplotlib-3.4.1-cp37-cp37m-manylinux1_x86_64.whl (10.3 MB)
  Downloading matplotlib-3.4.0-cp37-cp37m-manylinux1_x86_64.whl (10.3 MB)
INFO: pip is looking at multiple versions of cycler to determine which version is compatible with other requirements. This could take a while.
  Downloading matplotlib-3.3.4-cp37-cp37m-manylinux1_x86_64.whl (11.5 MB)
  Downloading matplotlib-3.3.3-cp37-cp37m-manylinux1_x86_64.whl (11.6 MB)
  Downloading matplotlib-3.3.2-cp37-cp37m-manylinux1_x86_64.whl (11.6 MB)
Requirement already satisfied: certifi>=2020.06.20 in /global/common/software/cmb/cori/cmbenv-gcc_20210527/cmbenv_python/lib/python3.7/site-packages (from matplotlib->healpy<2.0.0,>=1.14.0->mapsims==2.4.0) (2020.12.5)
INFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. If you want to abort this run, you can press Ctrl + C to do so. To improve how pip performs, tell us what happened here: https://pip.pypa.io/surveys/backtracking
INFO: pip is looking at multiple versions of certifi to determine which version is compatible with other requirements. This could take a while.
Collecting certifi>=2020.06.20
  Using cached certifi-2021.5.30-py2.py3-none-any.whl (145 kB)
  Downloading certifi-2020.12.5-py2.py3-none-any.whl (147 kB)
  Downloading certifi-2020.11.8-py2.py3-none-any.whl (155 kB)
  Downloading certifi-2020.6.20-py2.py3-none-any.whl (156 kB)
Collecting matplotlib
  Downloading matplotlib-3.3.1-cp37-cp37m-manylinux1_x86_64.whl (11.6 MB)
INFO: pip is looking at multiple versions of certifi to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of matplotlib to determine which version is compatible with other requirements. This could take a while.
  Downloading matplotlib-3.3.0-1-cp37-cp37m-manylinux1_x86_64.whl (11.5 MB)
  Downloading matplotlib-3.2.2-cp37-cp37m-manylinux1_x86_64.whl (12.4 MB)
  Downloading matplotlib-3.2.1-cp37-cp37m-manylinux1_x86_64.whl (12.4 MB)
  Downloading matplotlib-3.2.0-cp37-cp37m-manylinux1_x86_64.whl (12.4 MB)
  Downloading matplotlib-3.1.3-cp37-cp37m-manylinux1_x86_64.whl (13.1 MB)
INFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. If you want to abort this run, you can press Ctrl + C to do so. To improve how pip performs, tell us what happened here: https://pip.pypa.io/surveys/backtracking
  Downloading matplotlib-3.1.2-cp37-cp37m-manylinux1_x86_64.whl (13.1 MB)
  Downloading matplotlib-3.1.1-cp37-cp37m-manylinux1_x86_64.whl (13.1 MB)
  Downloading matplotlib-3.1.0-cp37-cp37m-manylinux1_x86_64.whl (13.1 MB)
  Downloading matplotlib-3.0.3-cp37-cp37m-manylinux1_x86_64.whl (13.0 MB)
  Downloading matplotlib-3.0.2-cp37-cp37m-manylinux1_x86_64.whl (12.9 MB)
  Downloading matplotlib-3.0.1-cp37-cp37m-manylinux1_x86_64.whl (12.9 MB)
  Downloading matplotlib-3.0.0-cp37-cp37m-manylinux1_x86_64.whl (12.8 MB)
  Downloading matplotlib-2.2.5-cp37-cp37m-manylinux1_x86_64.whl (12.8 MB)
Requirement already satisfied: pytz in /global/common/software/cmb/cori/cmbenv-gcc_20210527/cmbenv_python/lib/python3.7/site-packages (from matplotlib->healpy<2.0.0,>=1.14.0->mapsims==2.4.0) (2021.1)
  Downloading matplotlib-2.2.4-cp37-cp37m-manylinux1_x86_64.whl (12.8 MB)
ERROR: Operation cancelled by user

As you can see I finally killed it after 20 minutes.  I'll try again and let it run for longer, but it seems kind of crazy...

I'll probably remove poetry and use something more reasonable sometime in September or October.

The installation eventually died with an error trying to install pyparsing.  I then generated a setup.py from the pyproject.toml using a slightly modifed version of this script. The resulting setup.py had the same problem.  I think the issue is some kind of mutually exclusive version requirements of the dependencies.  Can you tell me what python version(s) you have used successfully with this?

In Github action it was working with 3.6 7 8

switched from poetry to flit in https://github.com/galsci/mapsims/commit/1942d3193bcab083f943fdd18f149fc25c5ea7fa
this should hopefully be fixed.


How to modify code with fixed pose？

Thanks for the great work. I would like to use real poses and fix poses  . How should I proceed and is it possible to achieve?  Any suggest will be very helpful, thank you!

If I understand correctly what you're trying to do - use actual measured poses as inputs along with the video frames, there is a way to do that.
In motion_filter.track(), there are two calls to self.video.append(). The first passes in a pose of Id to initialize the pose to identity for the first frame. The next passes in a pose of None to run the algorithm without pose input information. You will need to pass your poses into motion_filter.track() and then pass that in the self.video.append() call instead of Id or None.
You'll need to make sure your pose has been properly converted to SE3.


Old metrics in starboard-exporter due to starboard vulnerabilityreport per replicaset

Today it looks like we have much more CVE:s in our cluster then we actually do.
This thanks to how starboard generates a vulnerabilityreport per replicaset and the starboard-exporter gathers metrics from all the vulnerabilityreports.
It would be nice to solve this in starboard somehow but it will probably take some time. I have created a issue in https://github.com/giantswarm/starboard-exporter/issues/35 and hopefully they will be open to the idea.
When we manage to solve this somehow we should upgrade starboard or stoarboard-exporter to get better metrics.

Will manage this in: https://github.com/XenitAB/terraform-modules/issues/515
But it's solved in the new 0.14.0 release


bandstats issue

I am trying to run sumo but I get an error. I installed all the dependencies indicated in
https://sumo.readthedocs.io/en/latest/index.html
and I run any of the sumo the codes (sumo-*) in the VASP folder that includes all the standard output files after band structure calculations.
However, I obtain this error message:

Traceback (most recent call last):
File "/Users/Marco/opt/anaconda3/bin/sumo-bandstats", line 10, in
sys.exit(main())
File "/Users/Marco/opt/anaconda3/lib/python3.7/site-packages/sumo/cli/bandstats.py", line 326, in main
  parabolic=args.nonparabolic)
  File "/Users/Marco/opt/anaconda3/lib/python3.7/site-packages/sumo/cli/bandstats.py", line 93, in bandstats
    bs = vr.get_band_structure(line_mode=True)
  File "/Users/Marco/opt/anaconda3/lib/python3.7/site-packages/pymatgen/io/vasp/outputs.py", line 858, in get_band_structure
    if '' in kpoint_file.labels:
TypeError: argument of type 'NoneType' is not iterable------------------
I am wondering if you experienced this problem and maybe know how to solve it.
Thanks

You should only run sumo-bandstats on line mode band structure calculations. For example using KPOINTS files generated by sumo-kgen. You can see the documentation for using kgen here: https://sumo.readthedocs.io/en/latest/sumo-kgen.html
If your calculation WAS on a high-symmetry path, just not one generated by sumo, you can edit the KPOINTS file to manually include the k-point labels for the high symmetry points. For example, the line containing the gamma k-point:
0 0 0

Would be changed to:
0 0 0 \Gamma

Hi @fronzi  can you confirm if this solved your problem?

Hi @fronzi can you confirm if this solved your problem?

Hi @ajjackson , yes this solved the problem


Running target issue

Please, could you help me to understand: I defined default target "Report", but when I run the script (see below) Powershell shows error:

var target = Argument("Target", "Report");

Task("Start")
  .Does(() =>
{
  Information("Hello World!");
});

Task("Tests")
	.IsDependentOn("Start")
	.Does(()=>
{
	Information("Tests!");
});

Task("Report")
	.IsDependentOn("Tests")
	.Does(()=>
{
	Information("Report!");
});

RunTarget(target);

@constructor-igor have a look here:
https://github.com/cake-build/example/blob/master/build.ps1#L44
If you don't specify a target argument for the build.ps1 the default is Default. As a result, there is not a target called that in your script. Normally, I leave the Default task in place, but instead use an IsDependentOn method call to specify which Target should be invoked when calling Default.

but what does it mean var target = Argument("Target", "Report"); ?
I thought it defines default "Report"

It does, when there is no target being passed in. However, the bootstrapper is always passing a target, I.e. Default 🙂

ok. understood ;)
thank you,
Igor.


Website Down (Err 403)

The online app seems down due to a 403 Error:

https://gitattributes.io/

... full error message:

Error 403 - This web app is stopped.
The web app you have attempted to reach is currently stopped and does not accept any requests. Please try to reload the page or visit it again soon.
If you are the web app administrator, please find the common 403 error scenarios and resolution here. For further troubleshooting tools and recommendations, please visit Azure Portal.

Yes, I have issue with Azure Subscription and services running in it. I hope it will be resolved soon.

Up and running again.


Alert/Toast visualization adding proposal result

Creating an alert/toast to visualize the server response to the professor action of adding a new proposal

toast added


series request limit can be bypassed by users

Description
If you have a "Series Request Limit" set for a user, for example
4 Seasons per 7 days
The user is able to bypass the season limit when you have the "Auto Approve" setting disabled.
While it's in the "Pending State" awaiting admin approval, the user can go back in and edit the request and select more seasons than they are allowed.
Version
1.7.0
Steps to Reproduce

Create a test user
Assign a limit to this user for seasons. For example, allow 4 season requests per 7 days. Note: Assign a limit to the user using the Override Global Limit option, don't assign a limit to the user just using the global limit. (General > Users > Select User > General Tab > Series Request Limit > Override Global Limit)
Disable "Auto Approve" for this user, so that it requires admin approval.
On the test user, request a show that has more than 4 seasons, but select all that you're allotted to (4 seasons), Then submit
Once submitted, head to the requests tab, edit your request, then select the remaining seasons and click save.

Screenshots

Logs
No response
Platform
desktop
Device
Any
Operating System
Unraid
Browser
Any
Additional Context
No response
Code of Conduct

[X] I agree to follow Jellyseerr's Code of Conduct

Can you try on develop

Same issue

I am unable to recreate this issue on develop. When I try to edit the request, it expectedly still enforces the limit. Am I missing something?

I just updated to the latest develop and still see the issue, I don't have the "No season requests remaining" prompt when I edit a request, I am not sure how you have that
Are you overriding the global limit for your user? (Step 2 in the replication steps). I will update this in case there was confusion

I am unable to recreate this issue on develop. When I try to edit the request, it expectedly still enforces the limit. Am I missing something?

I've found the root setting that's causing this issue,  you have to have "Advanced Requests" enabled for the user as well. When I unchecked this permission from my test user, I see the series limit just as you would expect, per your screenshot. Once I check that option, I can bypass my series limit by editing the request.

Thank you for sharing that, it really helped
The issue seems to be originating from here. The value of quota is initially correct, but it immediately changes to null because of the requestOverrides?.user?.id condition becoming true
https://github.com/Fallenbagel/jellyseerr/blob/db84f6529ab285be26c96daaab065dfabf347417/src/components/RequestModal/TvRequestModal.tsx#L90-L95
Removing that entire condition worked for me, but I wanted to ask, is it there for a reason? Should we not be enforcing quota on every user regardless of these conditions?

Removing that entire condition worked for me, but I wanted to ask, is it there for a reason? Should we not be enforcing quota on every user regardless of these conditions?

Good question. I'll ask overseerr about this. I feel like advanced request is still not a user level that should get the quotas to be null

Removing that entire condition worked for me, but I wanted to ask, is it there for a reason? Should we not be enforcing quota on every user regardless of these conditions?

Good question. I'll ask overseerr about this. I feel like advanced request is still not a user level that should get the quotas to be null

Did they ever get back to you on this?

Yes. Not intended behavior. I have a fix, I'll be opening a pr on overseerr


Updating Waltz dependencies

Note - this change contains a jooq update
#4950

Adding everyone as a reviewer to remind you that you will need to obtain (and install) the latest version of jooq (if using sql server)


Add command-line python tools to assist with download

I think it would be good to have a command-line wrapper of the download API that would allow user to download content specified by either manifest or DICOM UIDs.
Inspiration for this suggestion was this snippet from @LennyN95 in this tutorial that utilizes s5cmd:
$ s5cmd --no-sign-request --endpoint-url https://s3.amazonaws.com \
  cp 's3://idc-open-data/8e4ccba1-74f4-4ffa-93f1-2a0263c8c9cf/*' $in && \
  s5cmd --no-sign-request --endpoint-url https://s3.amazonaws.com \
  cp 's3://idc-open-data/7271ab1d-bb03-4ca9-9457-dabcafd4d33f/*' $in && \
  s5cmd --no-sign-request --endpoint-url https://s3.amazonaws.com \
  cp 's3://idc-open-data/5541d783-4c8b-4d1b-8eae-99a868012082/*' $in && \
  echo "Downloaded $(ls $in | wc -l) files"

It should be very easy to have something like this, with download-from-idc script added to the path on package install
$ download-from-idc --destinationDir . \
  1.2.840.1136<IP_ADDRESS>425808326883186792123057288612355322 \
  1.2.840.1136<IP_ADDRESS>933508291954731667890414920476511765 \
  1.2.840.1136<IP_ADDRESS>144834924218414213677353968537663991

For the reference, this is how python command-line tools are implemented in pyradiomics: https://github.com/AIM-Harvard/pyradiomics/blob/master/radiomics/scripts/init.py.

to have a command-line wrapper

https://github.com/ImagingDataCommons/idc-index/pull/32 already provide a wrapper s5cmd as installed "python script".
Once it is integrated, I suggest the following:

Rename idc_index/index.py to idc_index/client.py
Add idc_index/cli.py with function like download_from() upload_to(), main() ... then you would just add the following to pyproject.toml:

[project.scripts]
s5cmd = "idc_index.s5cmd:s5cmd"
+download-from-idc = "idc_index.cli:download_from"
+upload-to-idc = "idc_index.cli:upload_to"
+ idc = "idc_index.cli:main"

You may want to revisit the "executable" name to be prefixed by idc
For argument parsing, I suggest to use click (See https://click.palletsprojects.com/), I can provide example as well.
The cli prarts, could also be optional and made available through idc-index[cli] ...

And once we have that command line tool, looks like it might be easy to make a GUI app around it: https://github.com/chriskiehl/Gooey.


⚠ Bofrost.at has degraded performance

In 3c313f7, Bofrost.at (https://www.bofrost.at) experienced degraded performance:

HTTP code: 200
Response time: 1145 ms

Resolved: Bofrost.at performance has improved in 0fafad2.


{{#each}} rerenders on every array set

Here is a reproduction.
Every keystroke in the text field causes the loop to rerender and the focus from the textfield to be lost. Check the log - it logs on every keystroke (meaning that the loop redraws).
Adding a key="@index" to the each fixes the problem but that's a "hack" I guess.
The problem is this line. However, removing it will probably cause other issues. So a clever solution should be thought of.

The same behavior occurs if you use standard EmberArray tracking/reactivity:
import { action } from '@ember/object';
import Controller from '@ember/controller';

export default class ApplicationController extends Controller {
  array = ['asd'];

  @action
  setValue(index, event) {
    this.array.replace(0, 1, [event.target.value]);
  }
}

This behavior makes sense, because the value to the each has changed. It needs to rerender each item, and the items are keyed off their identity, which in this case is the string itself. The string has changed, so the item has changed, and it is fully replaced.
You could either pass key="@index" to the each to have it key off the index instead, or you could wrap the string in an object whose identity is stable as you mutate the value.


[firebase_auth] Added Apple auth provider for iOS devices

Description
This PR enable the to use sign-in-ios to link with a firebase authentication account. Only run on iOS at the moment, but it's very important because Apple reject Apps in the stores wich implements other third-party login service but does not offer Sign in with Apple (Guideline 4.8 - Design - Sign in with Apple)
Related Issues
This PR resolve this issue in firebase_auth: Sign in with Apple #1434 for iOS.
https://github.com/FirebaseExtended/flutterfire/issues/1434
Checklist
Before you create this PR confirm that it meets all requirements listed below by checking the relevant checkboxes ([x]). This will ensure a smooth and quick review process.

[x] I read the Contributor Guide and followed the process outlined there for submitting PRs.
[x] If the pull request affects only one plugin, the PR title starts with the name of the plugin in brackets (e.g. [cloud_firestore])
[x] My PR includes unit or integration tests for all changed/updated/fixed behaviors (See Contributor Guide).
[x] All existing and new tests are passing.
[x] I updated/added relevant documentation (doc comments with ///).
[x] The analyzer (flutter analyze) does not report any problems on my PR.
[x] I read and followed the Flutter Style Guide.
[x] I updated pubspec.yaml with an appropriate new version according to the pub versioning philosophy.
[x] I updated CHANGELOG.md to add a description of the change.
[x] I signed the CLA.
[x] I am willing to follow-up on review comments in a timely manner.

Breaking Change
Does your PR require plugin users to manually update their apps to accommodate your change?

[ ] Yes, this is a breaking change (please indicate a breaking change in CHANGELOG.md and increment major revision).
[x] No, this is not a breaking change.

Ok, I could do it. But at now It's not a requirement in my project, I do this because Apple reject my publishment and I needed this change. I put this change in my workflow and I'll do it for Android in other PR. This PR it's only for iOS (I'm sure there is a lot of development that need this change)

I need to do 2 PR, becausa i have cross dependencies.
I close this to make a PR for firebase_auth_platform_interface before, and then, I return to make a new PR to solve this

I reopen this.
This PR have dependency of https://github.com/FirebaseExtended/flutterfire/pull/1549

Waiting for https://github.com/FirebaseExtended/flutterfire/pull/1549 merged

Some news?

@kleinpetr I'm working to do the changes in https://github.com/FirebaseExtended/flutterfire/pull/1549 to adjust the new specifications.
At now, I'm probe the new solutions.

@kleinpetr If you are impatient, you can use my repo as dependency unless resolve this.
firebase_auth:
git:
url: https://github.com/zariweyo/flutterfire.git
path: packages/firebase_auth/firebase_auth
ref: develop
or you car fork this. I'm use this in my project and run fine (remember, only in iOS)

Thank you so much man!!
čt 5. 12. 2019 v 15:16 odesílatel zariweyo<EMAIL_ADDRESS>napsal:

@kleinpetr https://github.com/kleinpetr If you are impatient, you can
use my repo as dependency unless resolve this.
firebase_auth:
git:
url: https://github.com/zariweyo/flutterfire.git
path: packages/firebase_auth/firebase_auth
ref: develop
or you car fork this. I'm use this in my project and run fine (remember,
only in iOS)
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/FirebaseExtended/flutterfire/pull/1526?email_source=notifications&email_token=AEDYEPKTORE25CXPS55JAQTQXEENDA5CNFSM4JTKGWN2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEGA3AMY#issuecomment-562147379,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AEDYEPNBN32FLOPZGWNJW5DQXEENDANCNFSM4JTKGWNQ
.

--
Tvorba webových a mobilních aplikací
E-mail<EMAIL_ADDRESS>Tel.: +420 606 954 642
IČO: 03771571
Web: www.kleinpetr.com

@zariweyo I am installed the package. But I cant find AppleAuthProvider class..

@kleinpetr I just commit a change in repo. The correct class is PlatformOAuthCredential, that is the change in the other PR.
The use case its like this
import 'dart:async';
import 'package:ehoreca_app/index.dart';
import 'package:ehoreca_app/module_common/index.dart';
import 'package:firebase_auth/firebase_auth.dart';
import 'package:apple_sign_in/apple_sign_in.dart';
import 'package:flutter/services.dart';
class Example{

  FirebaseAuth _firebaseAuth = FirebaseAuth.instance;

  Future<bool> signInApple({bool permitRegister=true}) async {
    Completer completer = new Completer<bool>();

    final Future<AuthorizationResult> _appleSignIn = AppleSignIn.performRequests([
      AppleIdRequest(requestedScopes: [
        Scope.email
      ])
    ]);

    _appleSignIn.then((AuthorizationResult result) async {
      switch(result.status) {
        case AuthorizationStatus.authorized:
          final AppleIdCredential appleIdCredential = result.credential;

            OAuthProvider oAuthProvider = new OAuthProvider(providerId: "apple.com");
            final AuthCredential credential = oAuthProvider.getCredential(
              idToken: String.fromCharCodes(appleIdCredential.identityToken),
              accessToken: String.fromCharCodes(appleIdCredential.authorizationCode),
            );

            try {
              final AuthResult _res = await _firebaseAuth.signInWithCredential(credential);
              completer.complete(_res.user.uid);
            } catch(err) {
                completer.completeError(err);
            }
            break;
        case AuthorizationStatus.error:
          completer.completeError(GenericError(
            code:"APPLE_AUTHORIZED_SIGN_IN_ERROR",
            message: result.error.localizedDescription,
            origin: LoginController
          ));
          break;
        case AuthorizationStatus.cancelled:
          completer.completeError(GenericError(
            code:"USER_CANCELED_LOGIN",
            message: "User has canceled the login",
            origin: LoginController
          ));
          break;
      }
    }).catchError((err) {

        if(err is PlatformException){
          completer.completeError(GenericError(
            code:err.code,
            message: err.message,
            origin: LoginController
          ));
        }else{
          completer.completeError(GenericError(
            code:"APPLE_SIGN_IN_ERROR",
            message: err.toString(),
            origin: LoginController
          ));
        }

        UtilsFunctions.onErrorEvent(err);
    });

    return completer.future;
  }

}

Maybe the title now It's more correct

@zariweyo
Thank you, this is what I need
OAuthProvider oAuthProvider = new OAuthProvider(providerId: "apple.com");
           final AuthCredential credential = oAuthProvider.getCredential(
             idToken: String.fromCharCodes(appleIdCredential.identityToken),
             accessToken: String.fromCharCodes(appleIdCredential.authorizationCode),
           );

Do you will implement another providers than Apple?

I will implement a version for Android, because I need this in my project too, but It's less critical (I first need to colaborate to change the apple_sign_in package to work with Android ).
And this solution really it's for any provider which can authenticate with Oauth. This is a generic solution now (I change the title for it), but my motivation was the Apple provider.

I'm using your temporal repo and I'm getting these errors:
No known class method for selector 'credentialWithProviderID:IDToken:rawNonce:accessToken:'
No known class method for selector 'credentialWithProviderID:IDToken:rawNonce:'

Yes, this is because you need to clean a reinstall your pods. Maybe clean cache of pods too. Make sure to use de Firebase/Auth 6.4
I had the same issue and in this PR I will update that requirement

How do I upgrade Firebase/Auth to 6.4? I did reinstall the Pods and even pod update and the latest version is 6.3

Perfect. You are rigth, the last version is 6.3.1. I will update with that.

Does this PR also make firebase_auth apple_sign_in available for flutter web?

Does this PR also make firebase_auth apple_sign_in available for flutter web?

Not yet, but adding support can be done in a subsequent PR; see _getCredential.

I dont know but now I am getting this error: The operation could not be completed....

@kleinpetr this is an error in apple_sign_in, not in flutterfire package. This PR is only for link credentias in firebase authentication.
Please, go to https://github.com/tomgilder/flutter_apple_sign_in to report your error. (for advance, remember that Apple sign is only available in iOS13 and accounts with double factor activated)

Yes, I'll do It for android, but before I need ti solve the sign om Android. At now, that package not working un Android.
I could modify java plugin in the same line as the objetivec, but l can't trying It.
In my project I need the Android versión too, at now, I only show the Apple sign in iOS. But I have other issues to solve before.
You decide. Do I modify the Android library without trying it?.

If you want to take a crack at adding an implementation, I'll look at what is possible testing-wise. We should have an integration test that covers both iOS and Android ideally (though it doesn't need to specifically connect to Apple).

You are right. I did implement the Android version, and I probe with google provider.

  Future<bool> signInGoogle(GoogleSignInAuthentication googleAuth) async {
    Completer completer = new Completer<bool>();
    OAuthProvider oAuthProvider = new OAuthProvider(providerId: "google.com");
    final AuthCredential credential = oAuthProvider.getCredential(
      idToken: googleAuth.idToken,
      accessToken: googleAuth.accessToken,
    );

    try {
      final AuthResult _res = await _firebaseAuth.signInWithCredential(credential);
      completer.complete(true);
    } catch(err) {
        completer.completeError(err);
    }

    return completer.future;
  }

It works perfectly. So I'm going to change the PR title. Now this is a generic solution of OAuth authentication for Android and iOS.

Everything is working perfectly 👍 . I am using this branch in production. When will this PR be merged?

I hope soon. This last change I think that is the complete solution.

Thank you for this PR.
When will this PR be merged?

Now everything is done, right?

@zariweyo Getting a PlatformExeption ERROR_INTERNAL_ERROR

@EliasDeuss this PR is closed. Please, report that issue yo the article's author.


correct entry G029

In galaxy_database.csv, G029: Correct Galaxy Type from 'Elliptical_l' to 'Spiral'.

I wan to work on this problem.


WIQL reference text describes a more complex query than shown

Disclaimer: I'm still learning WIQL for use in Azure DevOps queries, and I'm trying to learn more because I'm constantly running into issues with queries involving dates.
In the section covering date macros in the WIQL reference syntax doc, there is this line…

This syntax allows you to nest modifiers and offset your query twice. For example, the following clause filters work items that have been closed last year and three months into the start of the current year.

…and subsequent query code block (Markdown lines)…

[Microsoft.VSTS.Common.ClosedDate] >=@StartOfYear('+3M') - 1

To me, that text seems to describe a query that would require two clauses, while the displayed query is only a single clause. And I'm not totally sure the given query correctly describes one half of such a query.
Part of the problem could be that these StartOf* macros don't seem documented enough to remove ambiguity with their interpretation/execution.
Possible solution: query unchanged but text changes
If could just be a typo in the text. If, instead, the text said, "…work items that have been closed since four months into last year", the query example might be completely accurate as it is.
Again, I could be interpreting these macros incorrectly, but here's my interpretation of this query assuming the current year is 2020.

@StartOfYear() would be 01 January 2020
So, adding ('+3M') would add three months to the start of the current year: 01 April 2020
Then, adding  - 1 would take away another entire year: 01 April 2019

So, any issues closed >= that date (AKA "more recent than that date") would be caught by that query.
Possible solution: text unchanged but query gets more complex
Keep the text the same but the query needs updating. The text description implies a query that requires two clauses with an implied "between":

filters work items that have been closed [between] last year and three months into the start of the current year

I'm not sure it's worded quit sufficiently, but in my head, that would be "closed since last year" AND "closed…[up to] three months into the start of the current year".
If my understanding of WIQL is sufficient, that could be "closed since the start of last year but not closed after March of this year".
[Microsoft.VSTS.Common.ClosedDate] >= @StartOfYear() - 1
AND [Microsoft.VSTS.Common.ClosedDate] < @StartOfYear('+3M')

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 710af5f2-8a31-e0d3-4f20-00c79dc54713
Version Independent ID: 1d4d9580-984c-9fee-861a-a02ce828d3db
Content: WIQL reference syntax - Azure Boards
Content Source: docs/boards/queries/wiql-syntax.md
Product: devops
Technology: devops-agile
GitHub Login: @KathrynEE
Microsoft Alias: kaelli

@patridge -  Thank you for this feedback. I've added an item to my backlog and look forward to updating this article within the next few weeks.


Slow inference time using CPU

Hi All,
How are you?
Thank you for your valuable and amazing contribution.
I tried to inference whisper model large-v2 version with this repo and it seems like the inference time is pretty slow. I looked at some of the posted issues and benchmarks and saw that it should be lightning fast so obviously I'm doing something wrong.
Here is the code snip I used:
import time

from faster_whisper import WhisperModel

model_size = "large-v2"

# Run on GPU with FP16
# model = WhisperModel(model_size, device="cuda", compute_type="float16")

# or run on GPU with INT8
# model = WhisperModel(model_size, device="cuda", compute_type="int8_float16")
# or run on CPU with INT8
model = WhisperModel(model_size, device="cpu", cpu_threads=8, compute_type="int8")

for _ in range(10):
    stime = time.time()
    segments, info = model.transcribe(
        "data/foo_foo_sample.wav",
        beam_size=5,
        initial_prompt="focus on the word But."
    )
    print(f"transcribe time took: {time.time() - stime}")

print("Detected language '%s' with probability %f" % (info.language, info.language_probability))

for segment in segments:
    print("[%.2fs -> %.2fs] %s" % (segment.start, segment.end, segment.text))

I ran it over multiple short samples <10 sec each using two different systems with int_8 compute_type:

Mac M2  with 32gb ram -> took around 5 sec to transcribe.
EC2 r3.4xlarge machine with Linux OS, 16 CPU cores, and 120gb ram -> took around 10 sec to transcribe

Note that changing the passed arguments did not help either (cpu_threads, beam_size, etc.)
Thank you in advance!

Hi,
First, you don't measure the inference time correctly. You should include the loop for segment in segments in the inference time. See the warning in the README about segments being a generator: https://github.com/guillaumekln/faster-whisper#usage
Second, what are you comparing the inference time against? It can only be slow relative to something else (e.g. slower than openai/whisper or whisper.cpp).

Thank you for responding.
So if I follow you the total inference time would be even greater than what was reported in my previous message.
I did not make any comparison, but I saw the benchmarks provided in the README in addition to some posted in previous issues. Is receiving ~5 / ~10 sec timing (not including inference according to your explanation) for less than 3 sec file reasonable?

You may need to reduce the threads to 4. Even on my MacBook, threads greater than 4 resulted in slower results.

@archive-r  reducing from 8 to 2 resulted in doubling the time.
It is not clear to me why calling transcribe takes so long since it's not performing the inference itself.

https://github.com/guillaumekln/faster-whisper/issues/133#issue-1660690455

@archive-r reducing from 8 to 2 resulted in doubling the time.
It is not clear to me why calling transcribe takes so long since it's not performing the inference itself.

what is the sampling rate of the WAV file you used for your test?

@archive-r reducing from 8 to 2 resulted in doubling the time.
It is not clear to me why calling transcribe takes so long since it's not performing the inference itself.

what is the sampling rate of the WAV file you used for your test?

16 kHz

model.transcribe will run the model once to detect the language. In that case the time you measured is mostly expected. The large-v2 model is very expensive to run on the CPU.
So the inference time may be slow relative to your expectation, but it's still much faster than openai/whisper for example. You should use a smaller model if it's too slow for your usage.

@guillaumekln Thanks.
What if I have prior about the language? will it help to make it run faster?

Sure, the call to model.transcribe will be faster if you set a language. You can set the language code with model.transcribe(..., language="en").

Indeed, it's quite slow. A 2-minute MP3 takes 6 minutes to finish. Meanwhile, OpenAI's Whisper finishes within 2 minutes. Did I do something wrong? Here's the code I used:
        model = WhisperModel("small.en", device="cpu", compute_type="int8")
        segments, info = model.transcribe(self.src_filename, word_timestamps=True, beam_size=5,language="en")


Only return nodeinfo data if site is public

Give nodeinfo services the same indexing privileges as web crawlers based on the wordpress setting blog_public :

Discourage search engines from indexing this site

If the option is enabled, the nodeinfo path returns a 404 error as per the protocol

Background: My staging site somehow ended being heavily crawled by friendica servers.
@pfefferle Any objection to me merging this this?

I like the change! Will merge that later today and try to also release a new version!
Thanks a lot!

But I am not sure, this will solve the crawler problem.

True, I will monitor my logs to see if it is effective.
Thanks!