[storage-resize-images] Error from spawned child processes

Describe your configuration

Extension name: storage-resize-images
Extension version: 0.1.5
Configuration values:

Cloud Functions location: us-central1
Cloud Storage bucket for images: <my_project>.appspot.com
Sized of resized images: 260x180,520x360,1040x720
Deletion of original file: false
Cloud Storage path for resized images: thumbnails
Cache-Control header for resized images: max-age=86400

Describe the problem
When I upload a file to the storage bucket manually via the Firebase console, the extension's Cloud Function generates an error from each spawned child process.

Cloud Functions Logs

Child Process Error Details
Each child process gives the same general error, barring the specific image size and temporary filenames.
Error when resizing image { ChildProcessError: `convert /tmp/blb17228_49536908166_o_v02.0.jpg -resize 260x180> /tmp/blb17228_49536908166_o_v02.0_260x180.jpg` failed with code 1
    at ChildProcess.<anonymous> (/srv/node_modules/child-process-promise/lib/index.js:132:23)
    at emitTwo (events.js:126:13)
    at ChildProcess.emit (events.js:214:7)
    at maybeClose (internal/child_process.js:915:16)
    at Socket.stream.socket.on (internal/child_process.js:336:11)
    at emitOne (events.js:116:13)
    at Socket.emit (events.js:211:7)
    at Pipe._handle.close [as _onclose] (net.js:561:12)
  name: 'ChildProcessError',
  code: 1,
  childProcess:
   ChildProcess {
     domain:
      Domain {
        domain: null,
        _events: [Object],
        _eventsCount: 1,
        _maxListeners: undefined,
        members: [Array] },
     _events: { error: [Function], close: [Function] },
     _eventsCount: 2,
     _maxListeners: undefined,
     _closesNeeded: 3,
     _closesGot: 3,
     connected: false,
     signalCode: null,
     exitCode: 1,
     killed: false,
     spawnfile: 'convert',
     _handle: null,
     spawnargs:
      [ 'convert',
        '/tmp/blb17228_49536908166_o_v02.0.jpg',
        '-resize',
        '260x180>',
        '/tmp/blb17228_49536908166_o_v02.0_260x180.jpg' ],
     pid: 12,
     stdin:
      Socket {
        connecting: false,
        _hadError: false,
        _handle: null,
        _parent: null,
        _host: null,
        _readableState: [Object],
        readable: false,
        domain: [Object],
        _events: [Object],
        _eventsCount: 2,
        _maxListeners: undefined,
        _writableState: [Object],
        writable: false,
        allowHalfOpen: false,
        _bytesDispatched: 0,
        _sockname: null,
        _pendingData: null,
        _pendingEncoding: '',
        server: null,
        _server: null,
        _idleNext: null,
        _idlePrev: null,
        _idleTimeout: -1,
        [Symbol(asyncId)]: 1080,
        [Symbol(bytesRead)]: 0 },
     stdout:
      Socket {
        connecting: false,
        _hadError: false,
        _handle: null,
        _parent: null,
        _host: null,
        _readableState: [Object],
        readable: false,
        domain: [Object],
        _events: [Object],
        _eventsCount: 3,
        _maxListeners: undefined,
        _writableState: [Object],
        writable: false,
        allowHalfOpen: false,
        _bytesDispatched: 0,
        _sockname: null,
        _pendingData: null,
        _pendingEncoding: '',
        server: null,
        _server: null,
        _idleNext: null,
        _idlePrev: null,
        _idleTimeout: -1,
        write: [Function: writeAfterFIN],
        [Symbol(asyncId)]: 1081,
        [Symbol(bytesRead)]: 0 },
     stderr:
      Socket {
        connecting: false,
        _hadError: false,
        _handle: null,
        _parent: null,
        _host: null,
        _readableState: [Object],
        readable: false,
        domain: [Object],
        _events: [Object],
        _eventsCount: 3,
        _maxListeners: undefined,
        _writableState: [Object],
        writable: false,
        allowHalfOpen: false,
        _bytesDispatched: 0,
        _sockname: null,
        _pendingData: null,
        _pendingEncoding: '',
        server: null,
        _server: null,
        _idleNext: null,
        _idlePrev: null,
        _idleTimeout: -1,
        write: [Function: writeAfterFIN],
        [Symbol(asyncId)]: 1082,
        [Symbol(bytesRead)]: 357 },
     stdio: [ [Object], [Object], [Object] ] },
  stdout: '',
  stderr: 'convert-im6.q16: delegate failed `\'dwebp\' -pam \'%i\' -o \'%o\'\' @ error/delegate.c/InvokeDelegate/1919.\nconvert-im6.q16: unable to open file `/tmp/magick-126_p4YK94jmxX\': No such file or directory @ error/constitute.c/ReadImage/544.\nconvert-im6.q16: no images defined `/tmp/blb17228_49536908166_o_v02.0_260x180.jpg\' @ error/convert.c/ConvertImageCommand/3258.\n' }

The nature of these errors implies an environment/dependency problem with the extension.
Steps to reproduce:

Install the extension with the settings described above.
Upload an image to the root of the cloud storage bucket.

I'm not sure if it's relevant, but my Firebase project already contains a number of custom Cloud Functions, and they all use Node 10. This extension's installed Cloud Function uses Node 8.
Expected result

Three thumbnail images should be generated.

Actual result

No thumbnail images are generated.

Hi @andre-arsenault, the Firebase Extensions team will be publishing a release for the storage-resize-images library at some point in the very near future. It will be changing the current implementation using ImageMagick to Sharp library. I suspect this error will not occur anymore as the spawned child process will be removed. Here's the PR: #207 if you would like to track.

Thanks, I'll monitor that PR and wait for the next release.

Thank you @russellwheatley, I used your PR to write my own cloud function instead of waiting for it to be merged and published. I'm up and running now.


VersionBot: Nice to have: Block normal merges

At least once I've forgotten about versionbot, and merged PRs manually, leading to an annoying manual version update process. It'd be nice to versionbot took total responsibility of PRs (except for no-checks), and blocked normal merges completely, so that I can't accidentally usurp it and screw things up.

AFAIK there'd be no way for VersionBot to completely block normal merges, but at least on Etcher (which isn't using VersionBot yet) the repo is set up so that only the project-maintainer is allowed to merge PRs.

Still allows things to go wrong if you're the maintainer :smile:. The UI is set up to block merges if there's a failing status check, and Versionist does fail and block merges there when there's no change-type footer, it's just that it then unblocks them once there is. If they simply stayed blocked that'd solve this. Might have some other UX implications though (might mean that status markers on commits are always ❌?)

OTOH, preventing any "manual override" of VersionBot by the project-maintainer if / when things go wrong, might cause bigger problems? *shrug*

Several issues have now been seen in reverse, where people have hit the 'Merge' button without thinking and got master out of sync. This is a problem, and one solution is to hide the button under another status check that only resolves when the PR is approved. We actually want to remove the procbots/versionbot/ready-to-merge label (see #72).

 VersionBot: Nice to have: Give feedback on the changelog lines to be added in advance


Update the image lazy loader to support multiple data-src definitions from which it will randomly select one

Update the inline and in engrid-scripts image lazy loader to support multiple data-src definitions. When multiple are present it will pick one
Note that we'll also need to support matching Alt tags to go with the image.

No one actually wants this after they've had it in the past, closing this out


Variable for text on screen

Sorry of nubs question.
How I get date (time) var for display on screen?

I'm not sure if I understand it correctly.
Here's an example:
  CalendarDialog(
        state = rememberUseCaseState(visible = true),
        config = CalendarConfig(
            yearSelection = true,
            monthSelection = true,
            style = CalendarStyle.MONTH,
            disabledDates = disabledDates
        ),
        selection = CalendarSelection.Date { date ->
           // Your date is here
        },
    )

Check out the other documentation for more examples / samples.


ERROR: Remote recipe is newer than local recipe

Getting this with Jenkins, I thought I remember seeing that CPT had a force built in to override this error but I guess not.  Should it?
conans.errors.ConanException: Remote recipe is newer than local recipe:
 Remote date:<PHONE_NUMBER>
 Local date:<PHONE_NUMBER>

Need to add force=True here:
https://github.com/conan-io/conan-package-tools/blob/develop/cpt/uploader.py#L23

Yes, I will force it because it was forced before. but the real question is, why is it happening? do you have different times between slave machines?

yes, it looks like about 7 seconds difference.  This is a wider gap than it should be, but I would not say it's terribly uncommon for them not to agree.  Is there a threshold baked into the comparison?  For example, I would say that if they are less than 1 second apart, they're probably equivalent.  I guess the MOST robust thing to do (although I don't THINK I would recommend it), is for the conan client to compare client and server times at the start of the transaction and identify the offset between them.  Then, the server could always factor in that offset in the calculation. Anyway, we can close if you want.

Is there a reason that force=True is missing for Conan >= 1.8:
https://github.com/conan-io/conan-package-tools/blob/develop/cpt/uploader.py#L41-L45

With the usage of Conan 1.8 (with new upload policies in place) it shouldn't be necessary if only the timestamp is changing it will upload it anyway.
Is it breaking something on your side?

No thanks, I just ran into these line ending issues...

I'm experiencing this error from my Jenkins:
----------------------------------------
ERROR: Remote recipe is newer than local recipe:
 Remote date:<PHONE_NUMBER>
 Local date:<PHONE_NUMBER>

The upload seems to be proceeding, but it causes my Jenkins job to appear to have failed.
How would I make use of this force=True workaround?

This this ticket is closed, I've raised a new:
https://github.com/conan-io/conan-package-tools/issues/312 "ERROR: Remote recipe is newer than local recipe (cont.)"

I would try to explain what is this issue about and why we don't force the uploads by default.
Conan, when you upload a recipe or a package, check the manifest containing the checksums of the recipe files (usually just a conanfile.py). If the contents are the same, Conan doesn't care about the dates, because, actually will skip the real upload as it is already there.
But if the recipe is different, by default, we want to warn the user if you are trying to overwrite a newer recipe. How can it happen? well, if a slave takes less time to build a package than others, it could happen that the recipe uploaded is newer than the local one.
But why are they different? Usually, this is an issue with the CRLF characters. If you see some traces like the following you have the answer:
Local 'conanfile.py' using '\r\n' line-ends
Remote 'conanfile.py' using '\n' line-ends

Otherwise, if they are different, maybe you are mixing or overriding on purpose different recipes in the CI? Several consecutive pushes? Should it be happening? These are some questions to clarify before take the decission of forcing an upload.
Please, shed some light about you use case and we could enable a configuration to opt-in the package overwritting.
@michaelmaguire you mentioned a promotion process, could you explain a bit more?

you mentioned a promotion process, could you explain a bit more?

It's nothing too fancy -- we have a locked down Artifactory that only the official build user can published to.  Our Jenkinsfile for this 'promotion' project accepts as parameters repo and branch, and tries to run
conan create .. -s build_type=Release
conan create .. -s build_type=Debug
conan upload --all

It's single Jenkins pipeline job, running one job at a time.
I use:
        node {
            label 'WINDOWSBUILDSERVERS'
        }

to select a single node that will be used for the whole job, so not sure where any concurrency issues would come from.

Package is up to date, upload skipped

----------------------------------------
Remote manifest:
Time: 2018-12-11 10:28:20
conanfile.py, MD5: 611604d1f5e7629539c1935e83bc985a
export_source/CMakeLists.txt, MD5: 44393e04f71ac15b6d46280aaba7528a

Local manifest:
Time: 2018-10-10 08:50:31
conanfile.py, MD5: 762f77d5fde6a5421c4abe8c491ecfba
export_source/CMakeLists.txt, MD5: 44393e04f71ac15b6d46280aaba7528a

Local 'conanfile.py' using '\n' line-ends
Remote 'conanfile.py' using '\n' line-ends

----------------------------------------
ERROR: Remote recipe is newer than local recipe:
 Remote date:<PHONE_NUMBER>
 Local date:<PHONE_NUMBER>

Local manifest:
Time: 2018-10-10 08:50:31

Looks like an old version of the package. The conanfile is changing. Maybe you should do a diff between the remote and the "local" and see which is the good one. You could always remove the remote one if it is not the correct one.

This is bizarre.  I only Conanized this library early this morning.  There have been no previous versions of it.
I've verified the actual build node that was used for this build and checked the date on that build server:

C:\Users\mmaguir1>date
The current date is: Fri 12/21/2018
Enter the new date: (mm-dd-yy)

The only other machine I've ever built this on is my local dev server:

E:\DevelopBp\mmaguir1\src\nx\AcquisitionSupport>date
The current date is: Fri 12/21/2018

Very weird. That date is calculated in the "export" (also part of the "create")

Ahh.  Wait.  I think I've just discovered how I was mistaken.
The error in this cases wasn't coming from the package I tried to build for the first time.  It was from one of the other packages that was downloaded to enable the desired build.
I have this in my Jenkinsfile:
                    // Make sure we have nothing in our local cache as we push everything
                    // in our cache up.
                    // TODO: figure out how we can only upload exactly what we built below,
                    // so we don't have to nuke the cache each time.
                    conanClient.run(command: 'remove --force *')

                    // Old, manual way
                    //def buildInfo = conanClient.run(command: 'create . PORT/stable -s compiler="Visual Studio" -s compiler.version=14 -s compiler.toolset="v140_xp" -s build_type=Debug')
                    //conanClient.run(buildInfo: buildInfo, command: 'create . PORT/stable -s compiler="Visual Studio" -s compiler.version=14 -s compiler.toolset="v140_xp" -s build_type=Release')

                    // Nice way using configured profiles
                    def buildInfo = conanClient.run(command: 'create . PORT/stable --profile vs2015_14_v140_xp_Debug')
                    conanClient.run(buildInfo: buildInfo, command: 'create . PORT/stable --profile vs2015_14_v140_xp_Release')

                    // Upload everything we just build from our local cache.
                    String command = "upload * --all -r ${remoteNameConanNxStable} --confirm"

which I think is probably a bad shortcut.

The cache should be empty each time, I think the plugin creates a dedicated cache per build, so the remove might be unnecessary.

OK, I can remove the remove, but is there a better way to selectively upload only the built from source package?

Not directly but let me try a thing.

In this example I use the --json argument to conan create to get the created reference with an auxiliary function. I will add this to the Conan => integrations => jenkins:
def created_ref(a_json){
    def ref
    a_json["installed"].each { item ->
        def exported = item['recipe']['exported']
        if(exported){
            ref = item["recipe"]["id"]
            return ref
        }
    }
    ref
}

node("Linux") {
    sh "docker pull conanio/gcc7"
    docker.image("conanio/gcc7").inside {
        def server = Artifactory.server "ArtifactoryConan"
        def client = Artifactory.newConanClient()

        stage("Get project"){
            git branch: "release/1.2.11", url: "https://github.com/conan-community/conan-zlib.git"
        }

        stage("Create"){
            client.run(command: "--version")
            client.run(command: 'create . PORT/stable --json create.json')
            def my_json = readJSON file: "create.json"
            def ref = created_ref(my_json)
            echo "REF INSTALLED => ${ref}"
            //client.run(command: 'upload ${ref} --all -r ${remoteNameConanNxStable} --confirm')

        }
    }
}

Note: It requires this plugin: https://plugins.jenkins.io/pipeline-utility-steps

So, my problem seems to be the same problem as from conan-io/conan#4001

@lasote thanks, this --json and created_ref worked like a treat!

Question:
Can I ask above why we used:
        def exported = item['recipe']['exported']

and not, e.g.:
        def built = item['recipe']['built']

?

Sure, if you are building more than one pick all you need.

Hi, I have the same problem with conan 1.27.1 and conan_packages_tools 0.34.0 when uploading the package to our remote.
ERROR: Lame/3.100-34@****: Upload recipe to '****' failed: Remote recipe is newer than local recipe:
 Remote date:<PHONE_NUMBER>
 Local date:<PHONE_NUMBER>

Hi @NealRame, please, read https://docs.conan.io/en/latest/faq/using.html#packages-got-outdated-when-uploading-an-unchanged-recipe-from-a-different-machine
You are not alone, this problem is well known: https://github.com/conan-io/conan/issues/1400
Please, open a new issue in the future.

@uilianries thank you :-)


Streaming output support?

Wondering if streaming output is supported? Or are there any results about the time to first token and time per output token? Thanks.

@NiYueLiuFeng Can you share your modeling_internvl_chat.py？ Thank you very much


Added callback to history.replace().

Implements the feature request made in #698, specifically the idea I explained in my comment there.
I'm not entirely sure if the callback should also be added to other actions. A maintainers input either here or on the original feature request is greatly appreciated.
(Also: I can undo the white-space changes, as they create quite a lot of noise. But it's actually more consistent now)

I'm hesitant to add more API to this package, because it's already far too big. I think the root issue in #698 is that the location change happens asynchronously instead of synchronously like the browser does it. I think if we fix that (probably in a major bump) we can avoid adding more API to deal with the async behavior.

@mjackson: is there active work on a 5.0 version? The roadmap issue is locked, so I can't comment on there.
If 5.0 is on the horizon, that's great news (and I'll close this PR). But if it's still a while off then this feature might help to bridge the gap for those in need. We can still rip it out in 5.0 anyway, if we properly communicate that it's a temporary solution til synchronicity is restored.

Everything on master is going into 5.0. I'm planning on releasing a beta this week. Just need to button up a few details. I'll update the roadmap.

Everything on master is going into 5.0. I'm planning on releasing a beta this week.

Scratch that. I'm working on 5.0 in the dev branch. Development on 4.x will continue on master.


Purity not exceeding 79.1% (plus other strange behaviour)

Custom modpack with deepresonance-1.1.2 and mcjtylib-1.8.0
I have replicated the setup pictured in the tutorial. I put in a stack and a bit of ore and currently have 17400 or 17600mb of liquidcrystal circulating around but it's his 79.1% purity and is going no further. The purifier is no longer consuming filter material and there's plenty of space in the output chest.
OK this is weird, the volume in the tank is falling, filters still aren't being consumed, no spent filters being put in the chest, purity is not increasing. Tank below the valve (which is set to 84%) is still showing zero. No progress in the crystalizer, no crystals in the pedestal.

Can you show a screenshot of your setup?
You say the amount of liquid in the tank goes down?

http://i.imgur.com/rTvQhuu.png
The liquid volume slowly increased while I was feeding new ore into the smelter, then was stable at 17600 as expected once I stopped feeding it. I let it run for a while and the purity slowly ticked up as expected. It hit 79.1% and stayed there. I emptied the chests so it would be super-clear if the purifier was doing anything but more than 30 mins later, still nothing higher.
I broke and replaced a couple of the tanks, being sure to set the flow in the correct direction, then broke and replaced the purifier and the valve. At some point in there, the fluid level started dropping. Now it's a flat zero according to WAILA. It looks empty but there's still an occasional visual flicker of reddy-orange in one of the 5 tanks.
If you like, I can PM you on reddit and auth you to log into the server; it's a custom pack on technic.

To clarify, as I was writing the original post in this issue, I noticed the liquid level had started dropping. It progressively dropped but 100-200mb every couple of seconds until zero.

I will check it out. I'm going to release a new version of DR probably later this week and I'll try to fix this for that version.

OK! Thanks!

BTW, there seems to be a bug but one question. Is your RCL 100% quality? Because if it is less then 100% quality then the maximum purity you can achieve will also be lwess

Fixed in next release

Quality was over 90%, I think around 95-97% I was cooking it a little too quickly at first. Thanks for the rapid service, once again!

Ok, if quality is not 100% that explains why you could not get to 85% purity. Later today I will release a new version of DR with the bug fixed that the purifier destroys liquid

Updated to 1.1.3 and rebuilt the same setup. Siphoned off lower quality juice until I just had the good stuff. Took a long time (10+ mins?) to get from 84.9 to 85% but I now have 22 buckets of 85% pure fluid cycling around in the system.
Sadly the Valve doesn't seem to be passing anything through to the lower tank. I checked that the tank above is in output and the tank below is in input. Valve is set to 85 and 32000.
Incidentally, the "max in tank" defaults to zero. If that's likely to stop purified fluid passing unless set to something higher, you might want to change the default to prevent less-aware operators lodging tickets.

Try to set the valve to 84 instead. The values are not precise. Also clear the 32000 to be sure it is not affecting things. A value of 0 in that setting means that it will not check for maximum.

OK cool, fluid's flowing now and crystals are being generated, thanks.
I suspect that it was a design choice that this system not have one optimal setup, but I suggest that since the label says "Minimum percent purity to transfer", a value of 85 should let 85% fluid.
Thanks again for the rapid reply!


Unable to deploy broker skeleton

Hello, I am interested in using your starter-pack to implement a service broker in kubernetes. I followed the instructions and I am getting the following error:
ubuntu@cf-broker-master:~/go/src/github.com/pmorie/osb-starter-pack$ make deploy-helm
GOOS=linux GOARCH=amd64 CGO_ENABLED=0 \
go build -o servicebroker-linux --ldflags="-s" github.com/pmorie/osb-starter-pack/cmd/servicebroker
cp servicebroker-linux image/servicebroker
docker build image/ -t "quay.io/osb-starter-pack/servicebroker:125abe3"
Sending build context to Docker daemon  5.736MB
Step 1/3 : FROM busybox
 ---> 5b0d59026729
Step 2/3 : ADD servicebroker /opt/servicebroker/servicebroker
 ---> Using cache
 ---> a9422d727793
Step 3/3 : CMD /opt/servicebroker/servicebroker --help
 ---> Using cache
 ---> 06ed501e4358
Successfully built 06ed501e4358
Successfully tagged quay.io/osb-starter-pack/servicebroker:125abe3
helm upgrade --install broker-skeleton --namespace broker-skeleton \
charts/servicebroker \
--set image="quay.io/osb-starter-pack/servicebroker:125abe3",imagePullPolicy="IfNotPresent"
Error: UPGRADE FAILED: "broker-skeleton" has no deployed releases
Makefile:31: recipe for target 'deploy-helm' failed
make: *** [deploy-helm] Error 1

Seems like the build process works well but the install is not working. Not sure why is using image quay.io/osb-starter-pack/servicebroker:125abe3 instead of :latest
Any help is much appreciated.
Thanks!

Hi-
Thanks for using the repo!
Can I ask, are you deploying against a local cluster or one hosted somewhere off of your machine? If you're using a non-local, you'll need to push the image before it will be usable in the cluster you're deploying to.
You can set the image repository and tag used by setting the IMAGE and TAG environment variables respectively. You'll also need to use the make target to push the image.
example:
$ IMAGE=myimage TAG=latest make push deploy-helm

Hope that helps, feel free to ping me on kubernetes slack (nick pmorie) if you get stuck.

Thanks, @pmorie, that worked. After changing the IMAGE and TAG, I had an issue that it was not letting me deploy to kubernetes (even if the build and pushing the image was successful).  Had to run a helm delete --purge broker-skeleton. Maybe this could be added as a directive either under clean or make a new one clean-helm ?

Closing this issue. Updated documentation helped.


Add FFT

Integrate an FFT library to display frequency spectrum (or dominant frequency at least).

This could be easily implemented on a teensy using the teensy audio lib.

Thanks, good to know!


chore: stop testing on Kubernetes 1.23 and 1.24

Since version 1.23 and 1.24 are way out of support we should remove those version from the test matrix and also from the tested versions.
Closes #4535

/ok-to-merge updating supported versions


:fire: add footer html back into punchcard

Adding templating and html back into punchcard for the footer

@rachelnicole Please refer to our Emoji Cheatsheet. The fire emoji is for removing files. This is a bug fix, so it should be :bug:

@Snugug 👍


Select - caret not clickable if i add background to input

Description
Is this an issue with Materialize?
I add an white background to .select-dropdown class and z-index to .caret class. The caret is not clickable.
Any idea how to fix that?

https://codepen.io/gunnargj/pen/ZKpvYO

Can you provide a codepen with your code?

https://codepen.io/gunnargj/pen/ZKpvYO

So far I can see in the sourcecode, that the click is just caught on the input but not on the caret: https://github.com/Dogfalo/materialize/blob/master/js/forms.js
($newSelect is the input text field and it has focus and click listeners). You could add a click listener to the caret which calls .trigger('open') on the generated text input (traverse up using jQuery.closest()).
Normally the caret lies behind an element which catches the click event.

What about $select-background in the variables?
https://github.com/Dogfalo/materialize/blob/master/sass/components/forms/_select.scss
https://github.com/Dogfalo/materialize/blob/master/sass/components/_variables.scss

I tried this:
$( ".pickuplocation .caret" ).click(function() {
$( ".pickuplocation .select-dropdown" ).trigger( "click" );
return false;
});
That works, except when i click on something in the dropdown, it doesn't get selected

I also tried to change $select-background variable to #fff then the caret is hidden except when i add z-index to it and then it's not clickable

Seems we have to change this and also catch the click on the caret by default? Background be behind the caret imho.

Sounds good - do you have any workaround for me?

Currently not, I am not on my place.
@fega @tomscholz do you have some ideas for the best solution?

Fixed in c44a4af08902ae209fff690fd84f33041056d424


Flatpak unable to access account

It used to be that the Flatpak would detect my account from the system installed in Goldewarden. However, it is no longer doing this.
Furthermore, I can't even log in using the flatpak as I get stuck in this window and the login button does nothing

System goldwarden (AUR)
Name            : goldwarden
Version         : 0.2.9-1

Flatpak
Goldwarden          com.quexten.Goldwarden        0.2.9            stable       flathub     user

System
Operating System: Garuda Linux
KDE Plasma Version: 5.27.10
KDE Frameworks Version: 5.114.0
Qt Version: 5.15.12
Kernel Version: 6.7.0-3-cachyos (64-bit)
Graphics Platform: Wayland
Processors: 12 × AMD Ryzen 5 5600X 6-Core Processor
Memory: 31.3 GiB of RAM
Graphics Processor: AMD Radeon RX 6700 XT

Thanks for reporting, what is your $GOLDWARDEN_SOCKET_PATH set to?

Thanks for reporting, what is your $GOLDWARDEN_SOCKET_PATH set to?

[zany130@Garuda-Linux ~]$ echo $GOLDWARDEN_SOCKET_PATH

[zany130@Garuda-Linux ~]$

seems it is not set could be my issue, and I just saw this on the flatpak wiki page

Goldwarden is available as a graphical application as a Flatpak. By default, the Flatpak spawns a goldwarden daemon within the sandbox. You can also use an unsandboxed daemon with the Flatpak. In this case, make sure the daemon has the following environment variable set:
GOLDWARDEN_SOCKET_PATH=~/.var/app/com.quexten.Goldwarden/data/goldwarden.sock

Yeah, the path had to be changed when publishing to FlatHub. That's why the update broke your setup, sorry about that.


[regression in 1.2 - vs 1.0] com.google.cloud.tools.jib.maven.DecryptedMavenSettings requires all servers to be decryptable

Description of the issue:
in 1.0 only the used servers by the maven plugin was decrypted.
In 1.2, all are decrypted due to com.google.cloud.tools.jib.maven.DecryptedMavenSettings#from.
Expected behavior:
Get back 1.0 behavior.
Steps to reproduce:
Add a ciphered server not used by jib which can't be deciphered.
TIP: use an invalid password, in my case it was ${env.MISSING_VAR}
Environment:
Linux but probably any.
Log output:
Failed to execute goal com.google.cloud.tools:jib-maven-plugin:1.2.0:build (build) on project xxxx: Unable to decrypt settings.xml: [ERROR] Failed to decrypt password for server jetbrains: org.sonatype.plexus.components.sec.dispatcher.SecDispatcherException: java.io.FileNotFoundException: /home/travis/.m2/settings-security.xml (No such file or directory) @ server: jetbrains -> [Help 1]
(note: intentionally jib does not use settings.xml and it was working great in 1.0 but with 1.2 forced decryption makes the build failing, workaround is to pass decryption data but it is not desired)

Thanks for the bug report.
I think we should turn this into a warning instead of failing if decryption fails. ${env.MISSING_VAR} seems like a good example.

@chanseokoh personnally I wouldnt even decrypt servers which are not used to ensure clear values are not even loaded. Do you think it can be an option?

We also go through all the list of proxies to find active HTTP and HTTPS settings. Not failing on decryption failure will work as if nothing has been done for the failed decryption. It should just work, and I don't see any issue with it. What is your concern?

Also decryptions appears to be happening wholesale on the file at the moment using the maven api. I'm not sure exactly if it's possible to do this per server.

@chanseokoh having a build with warning can lead to a fail build on some CI platforms. One option can be to just ignore the failures but at the end there is never the need to decrypt something which is not used - which is safer by construction since we are speaking of passwords. Decryption can be done lazily when the server (same applies to proxy) is fetched injecting the component SettingsDecrypter as done here https://github.com/Talend/component-runtime/blob/87a387eb7796d11cab5ef4159035c54f1bbb3043/talend-component-maven-plugin/src/main/java/org/talend/sdk/component/maven/ImageM2Mojo.java#L195

I get what you're saying, but here's my argument.

(same applies to proxy)

If <proxies> are configured, all HTTP/HTTPS requests and responses should go through the proxies, so you'll end up having to decrypt the whole settings.xml anyways. Also, given that Maven API does the wholesale decryption and doesn't allow selectively decrypting individual passwords, I think there is no real concern about decrypting something unused. Sure, you could argue that if <proxies> are not ever been defined or there's no <password> for proxies, a lazy decryption for <servers> may make a difference, but given how Maven decrypts it, it doesn't seem worth another branch fork in our code.

having a build with warning can lead to a fail build on some CI platforms.

This is a rare situation in that you had ${env.FORGOT_TO_SET} in settings.xml and you forgot to define the said environment variable, and it may actually be better to point that out, especially for CI builds where people often have a lot of instances of failing to inject environment variables. Even if CI fails on warning, it's also an easy fix by setting the missing env variable.

@chanseokoh I'm not following:

maven does not enforce to decrypt the settings as a whole - it is actually rare in plugins.
About the "rare situation", it depends your settings.xml, locally I have ~20 credentials there and per project I tend to use 3-4 only. On the CI all PR builds will have this MISSING_VAR - not FORGOTTEN ;) - setup since part of the variables are only for master build - like documentation redeployment.

Ah, sorry. I didn't look into the Maven API actually. Seems like you can decrypt per server.

@rmannibucau, you're right, the API does allow it. We actually changed the code to wholesale decryption based on some weird code paths that we had (sharing gradle/maven code). We'll take another look at this to see if we can solve it more gracefully.

if it helps: a trivial way to solve it is to copy the settings (really new X().set(original.get()) pattern) and use a DecryptableServer instance extending Server which does the decryption lazily (you pass the decrypter in the constructor). It limits a lot the impacts even if not super elegant. That said happy with any solution on my side :)

Fixed by #1712. Closing.

@rmannibucau v1.3.0 is released, and it will not decrypt passwords that are only necessary on demand.


feat: adds --wait-after-healthy SECONDS allowing healthy container to…

This PR adds #27 with a new switch --wait-after-healthy SECONDS.
When specified and rollout can see 'health' of containers, it adds an additional sleep to give the new container time to settle down and/or a reverse proxy to detect that the new container is healthy.

Tested, seems to be working correctly. Thank you for the contribution!
Close #27


Add package building scripts.

Summary
Implement some basic scripts to facilitate the process of building and publishing the package.

Codecov Report

Merging #14 (6f465a4) into main (217e7d0) will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##              main       #14   +/-   ##
=========================================
  Coverage   100.00%   100.00%
=========================================
  Files           13        13
  Lines          962       962
=========================================
  Hits           962       962

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 217e7d0...6f465a4. Read the comment docs.


Enable configuration check for cast string to timestamp

We were seeing timestamp values returned as null on the GPU side when the CPU side returned valid data. It looks like we somehow weren't checking the conf we added which defaults to false when trying to convert strings to timestamps.
In this case the timestamp format was: 2017-11-29 20:00:35 which according to our docs we don't support. https://nvidia.github.io/spark-rapids/docs/compatibility.html#string-to-timestamp
I manually tested this for now:
    !Expression <Cast> cast(from_unixtime(value#0L, yyyy-MM-dd HH:mm:ss, Some(UTC)) as timestamp) cannot run on GPU because the GPU only supports a subset of formats when casting strings to timestamps. Refer to the CAST documentation for more details. To enable this operation on the GPU, set spark.rapids.sql.castStringToTimestamp.enabled to true.

build


PluginsConfiguration: send the right arguments to pre_check_and_set_p…

…latforms
it takes koji_target, not target.
Signed-off-by: Mark Langsdorf<EMAIL_ADDRESS>

Not updating release notes draft because this is bugfix to an unreleased feature.


some questions about using packet

I am glad to use your packet.However,when I install it with running command "./run.py -s sample/android/button/sample1.txt",I am faced with some question as followging:
07:36:13 merging 0 template(s)
07:36:13 semantics checking
Traceback (most recent call last):
File "./run.py", line 103, in
sys.exit(main())
File "./run.py", line 99, in main
return pasket.main(opt.cmd, opt.smpl, opt.tmpl, opt.pattern, opt.output)
File "/home/yh/pasket-master/lib/typecheck.py", line 384, in takes_invocation_proxy
return method(*args, **kwargs)
File "/home/yh/pasket-master/lib/typecheck.py", line 416, in returns_invocation_proxy
result = method(*args, **kwargs)
File "/home/yh/pasket-master/pasket/init.py", line 337, in main
decode.to_java(cmd, java_dir, tmpls, output_paths, _patterns)
File "/home/yh/pasket-master/lib/typecheck.py", line 384, in takes_invocation_proxy
return method(*args, **kwargs)
File "/home/yh/pasket-master/lib/typecheck.py", line 416, in returns_invocation_proxy
result = method(*args, **kwargs)
File "/home/yh/pasket-master/pasket/decode/init.py", line 236, in to_java
map(lambda vis: tmpl.accept(vis), _visitors)
File "/home/yh/pasket-master/pasket/decode/init.py", line 236, in
map(lambda vis: tmpl.accept(vis), _visitors)
AttributeError: 'NoneType' object has no attribute 'accept'
I tried to get corresponding params:
opt.cmd : android
opt.smpl : ['sample/android/button/sample1.txt']
opt.tmpl : []
opt.pattern : []
opt.output : ./result
I hope to hear from you!Tanking you so much!

Template, a.k.a. partial program, is a key input to pasket.  The goal of the tool is to find answers (we call it pattern instantiation) for pattern roles in the given template such that the synthesized model preserves the expected behavior, i.e., conforms to the given sample (call logs).  So, passing no tmpl means, you asked pasket to magically synthesize a model from scratch.  That would be great if we can do, but not yet. :)
Anyhow, maybe, our wordings in README are not well organized.  Please try this command instead:
./run.py -c android -p button

which will automatically set opt.smpl, opt.tmpl, and opt.pattern for you.

Thank you for your reply very much.I get the outout as follows:
android
[]
[]
['button']
./result
10
12:07:13 demo: button
12:07:14 rewriting accessor_adhoc pattern
12:07:14 rewriting accessor_map pattern
12:07:14 rewriting accessor_uni pattern
12:07:17 rewriting builder pattern
12:07:17 rewriting factory pattern
12:07:17 rewriting observer pattern
12:07:18 rewriting proxy pattern
12:07:18 rewriting singleton pattern
12:07:18 rewriting state pattern
12:07:18 semantics checking
12:07:18 building class hierarchy
12:07:48 encoding ./result/sk_button/type.sk
12:07:48 encoding ./result/sk_button/Object.sk
12:07:48 encoding ./result/sk_button/SymUtil.sk
12:07:48 encoding ./result/sk_button/Activity.sk
12:07:48 encoding ./result/sk_button/ActivityThread.sk
12:07:48 encoding ./result/sk_button/ComponentName.sk
12:07:48 encoding ./result/sk_button/Context.sk
12:07:48 encoding ./result/sk_button/ContextWrapper.sk
12:07:48 encoding ./result/sk_button/Intent.sk
12:07:48 encoding ./result/sk_button/Integer.sk
12:07:48 encoding ./result/sk_button/String.sk
12:07:48 encoding ./result/sk_button/StringBuffer.sk
12:07:49 encoding ./result/sk_button/StringBuilder.sk
12:07:49 encoding ./result/sk_button/BaseBundle.sk
12:07:49 encoding ./result/sk_button/Bundle.sk
12:07:49 encoding ./result/sk_button/Handler.sk
12:07:49 encoding ./result/sk_button/Looper.sk
12:07:49 encoding ./result/sk_button/Message.sk
12:07:49 encoding ./result/sk_button/MessageQueue.sk
12:07:49 encoding ./result/sk_button/SystemServer.sk
12:07:49 encoding ./result/sk_button/SystemServiceManager.sk
12:07:49 encoding ./result/sk_button/TelephonyManager.sk
12:07:49 encoding ./result/sk_button/TypedValue.sk
12:07:49 encoding ./result/sk_button/ContextThemeWrapper.sk
12:07:49 encoding ./result/sk_button/InputEvent.sk
12:07:49 encoding ./result/sk_button/KeyEvent.sk
12:07:49 encoding ./result/sk_button/MotionEvent.sk
12:07:49 encoding ./result/sk_button/View.sk
12:07:49 encoding ./result/sk_button/ViewGroup.sk
12:07:49 encoding ./result/sk_button/Window.sk
12:07:49 encoding ./result/sk_button/WindowManagerGlobal.sk
12:07:49 encoding ./result/sk_button/AbsSpinner.sk
12:07:49 encoding ./result/sk_button/AdapterView.sk
12:07:50 encoding ./result/sk_button/ArrayAdapter.sk
12:07:50 encoding ./result/sk_button/BaseAdapter.sk
12:07:50 encoding ./result/sk_button/Button.sk
12:07:50 encoding ./result/sk_button/CheckBox.sk
12:07:50 encoding ./result/sk_button/CompoundButton.sk
12:07:50 encoding ./result/sk_button/LinearLayout.sk
12:07:50 encoding ./result/sk_button/RadioButton.sk
12:07:50 encoding ./result/sk_button/RadioGroup.sk
12:07:50 encoding ./result/sk_button/RelativeLayout.sk
12:07:50 encoding ./result/sk_button/Spinner.sk
12:07:50 encoding ./result/sk_button/TextView.sk
12:07:50 encoding ./result/sk_button/Toast.sk
12:07:50 encoding ./result/sk_button/ButtonActivity.sk
12:07:50 encoding ./result/sk_button/Main.sk
12:07:50 encoding ./result/sk_button/R.sk
12:07:50 encoding ./result/sk_button/AuxAccessorMap.sk
12:07:55 encoding ./result/sk_button/AuxAccessorUni.sk
12:07:55 encoding ./result/sk_button/AuxObserverMotionEvent.sk
12:07:56 encoding ./result/sk_button/AuxSingleton.sk
12:07:56 encoding ./result/sk_button/sample_sample1.sk
12:07:56 encoding ./result/sk_button/log.sk
12:07:56 encoding ./result/sk_button/sample.sk
12:07:56 sketch running...
[1467356878.0230 - ERROR] [SKETCH] Sketch Not Resolved Error: nullThe sketch could not be resolved.
[1467356878.0280 - DEBUG] [SKETCH] stack trace written to file: ./result/tmp/stacktrace.txt
[1467356878.0281 - DEBUG] Backend solver input file at /home/yh/pasket-master/result/tmp/sk_button/input0.tmp
12:07:58 wrong modelings
I wonder if some configure I did wrong.And I noticed when I run "python -m grammar.gen" it shows a warning which I cannot solve:
java.g:691:9: Decision can match input such as "'0'..'9'{'E', 'e'}{'+', '-'}'0'..'9'{'D', 'F', 'd', 'f'}"
using multiple alternatives: 3, 4
As a result, alternative(s) 4 were disabled for that input
In addition,when I run "./run.py -c codegen" I am faced with a problem that "package sketch.complier.ast.core does not exist" . So I changed the build.xml in this:

...
...

I directly refer to the jar file.
All I changed are shown above,I am pleased to hope your reply,thanks so much.

Thank you for your reply very much. frustratingly I get the output as follows:
android
[]
[]
['button']
./result
10
12:07:13 demo: button
12:07:14 rewriting accessor_adhoc pattern
12:07:14 rewriting accessor_map pattern
12:07:14 rewriting accessor_uni pattern
12:07:17 rewriting builder pattern
12:07:17 rewriting factory pattern
12:07:17 rewriting observer pattern
12:07:18 rewriting proxy pattern
12:07:18 rewriting singleton pattern
12:07:18 rewriting state pattern
12:07:18 semantics checking
12:07:18 building class hierarchy
12:07:48 encoding ./result/sk_button/type.sk
12:07:48 encoding ./result/sk_button/Object.sk
12:07:48 encoding ./result/sk_button/SymUtil.sk
12:07:48 encoding ./result/sk_button/Activity.sk
12:07:48 encoding ./result/sk_button/ActivityThread.sk
12:07:48 encoding ./result/sk_button/ComponentName.sk
12:07:48 encoding ./result/sk_button/Context.sk
12:07:48 encoding ./result/sk_button/ContextWrapper.sk
12:07:48 encoding ./result/sk_button/Intent.sk
12:07:48 encoding ./result/sk_button/Integer.sk
12:07:48 encoding ./result/sk_button/String.sk
12:07:48 encoding ./result/sk_button/StringBuffer.sk
12:07:49 encoding ./result/sk_button/StringBuilder.sk
12:07:49 encoding ./result/sk_button/BaseBundle.sk
12:07:49 encoding ./result/sk_button/Bundle.sk
12:07:49 encoding ./result/sk_button/Handler.sk
12:07:49 encoding ./result/sk_button/Looper.sk
12:07:49 encoding ./result/sk_button/Message.sk
12:07:49 encoding ./result/sk_button/MessageQueue.sk
12:07:49 encoding ./result/sk_button/SystemServer.sk
12:07:49 encoding ./result/sk_button/SystemServiceManager.sk
12:07:49 encoding ./result/sk_button/TelephonyManager.sk
12:07:49 encoding ./result/sk_button/TypedValue.sk
12:07:49 encoding ./result/sk_button/ContextThemeWrapper.sk
12:07:49 encoding ./result/sk_button/InputEvent.sk
12:07:49 encoding ./result/sk_button/KeyEvent.sk
12:07:49 encoding ./result/sk_button/MotionEvent.sk
12:07:49 encoding ./result/sk_button/View.sk
12:07:49 encoding ./result/sk_button/ViewGroup.sk
12:07:49 encoding ./result/sk_button/Window.sk
12:07:49 encoding ./result/sk_button/WindowManagerGlobal.sk
12:07:49 encoding ./result/sk_button/AbsSpinner.sk
12:07:49 encoding ./result/sk_button/AdapterView.sk
12:07:50 encoding ./result/sk_button/ArrayAdapter.sk
12:07:50 encoding ./result/sk_button/BaseAdapter.sk
12:07:50 encoding ./result/sk_button/Button.sk
12:07:50 encoding ./result/sk_button/CheckBox.sk
12:07:50 encoding ./result/sk_button/CompoundButton.sk
12:07:50 encoding ./result/sk_button/LinearLayout.sk
12:07:50 encoding ./result/sk_button/RadioButton.sk
12:07:50 encoding ./result/sk_button/RadioGroup.sk
12:07:50 encoding ./result/sk_button/RelativeLayout.sk
12:07:50 encoding ./result/sk_button/Spinner.sk
12:07:50 encoding ./result/sk_button/TextView.sk
12:07:50 encoding ./result/sk_button/Toast.sk
12:07:50 encoding ./result/sk_button/ButtonActivity.sk
12:07:50 encoding ./result/sk_button/Main.sk
12:07:50 encoding ./result/sk_button/R.sk
12:07:50 encoding ./result/sk_button/AuxAccessorMap.sk
12:07:55 encoding ./result/sk_button/AuxAccessorUni.sk
12:07:55 encoding ./result/sk_button/AuxObserverMotionEvent.sk
12:07:56 encoding ./result/sk_button/AuxSingleton.sk
12:07:56 encoding ./result/sk_button/sample_sample1.sk
12:07:56 encoding ./result/sk_button/log.sk
12:07:56 encoding ./result/sk_button/sample.sk
12:07:56 sketch running...
[1467356878.0230 - ERROR] [SKETCH] Sketch Not Resolved Error: nullThe sketch could not be resolved.
[1467356878.0280 - DEBUG] [SKETCH] stack trace written to file: ./result/tmp/stacktrace.txt
[1467356878.0281 - DEBUG] Backend solver input file at /home/yh/pasket-master/result/tmp/sk_button/input0.tmp
12:07:58 wrong modelings
I wonder if some configure I did wrong.And I noticed when I run "python -m grammar.gen" it shows a warning which I cannot solve:
java.g:691:9: Decision can match input such as "'0'..'9'{'E', 'e'}{'+', '-'}'0'..'9'{'D', 'F', 'd', 'f'}"
using multiple alternatives: 3, 4
As a result, alternative(s) 4 were disabled for that input
In addition,when I run "./run.py -c codegen" I am faced with a problem that "package sketch.complier.ast.core does not exist" . So I changed the build.xml in this:

...
...

I directly refer to the jar file.
All I changed are shown above,I am pleased to hope your reply,thanks so much.

Thank you for your reply very much. frustratingly I get the output as follows:
android
[]
[]
['button']
./result
10
12:07:13 demo: button
12:07:14 rewriting accessor_adhoc pattern
12:07:14 rewriting accessor_map pattern
12:07:14 rewriting accessor_uni pattern
12:07:17 rewriting builder pattern
12:07:17 rewriting factory pattern
12:07:17 rewriting observer pattern
12:07:18 rewriting proxy pattern
12:07:18 rewriting singleton pattern
12:07:18 rewriting state pattern
12:07:18 semantics checking
12:07:18 building class hierarchy
12:07:48 encoding ./result/sk_button/type.sk
12:07:48 encoding ./result/sk_button/Object.sk
12:07:48 encoding ./result/sk_button/SymUtil.sk
12:07:48 encoding ./result/sk_button/Activity.sk
12:07:48 encoding ./result/sk_button/ActivityThread.sk
12:07:48 encoding ./result/sk_button/ComponentName.sk
12:07:48 encoding ./result/sk_button/Context.sk
12:07:48 encoding ./result/sk_button/ContextWrapper.sk
12:07:48 encoding ./result/sk_button/Intent.sk
12:07:48 encoding ./result/sk_button/Integer.sk
12:07:48 encoding ./result/sk_button/String.sk
12:07:48 encoding ./result/sk_button/StringBuffer.sk
12:07:49 encoding ./result/sk_button/StringBuilder.sk
12:07:49 encoding ./result/sk_button/BaseBundle.sk
12:07:49 encoding ./result/sk_button/Bundle.sk
12:07:49 encoding ./result/sk_button/Handler.sk
12:07:49 encoding ./result/sk_button/Looper.sk
12:07:49 encoding ./result/sk_button/Message.sk
12:07:49 encoding ./result/sk_button/MessageQueue.sk
12:07:49 encoding ./result/sk_button/SystemServer.sk
12:07:49 encoding ./result/sk_button/SystemServiceManager.sk
12:07:49 encoding ./result/sk_button/TelephonyManager.sk
12:07:49 encoding ./result/sk_button/TypedValue.sk
12:07:49 encoding ./result/sk_button/ContextThemeWrapper.sk
12:07:49 encoding ./result/sk_button/InputEvent.sk
12:07:49 encoding ./result/sk_button/KeyEvent.sk
12:07:49 encoding ./result/sk_button/MotionEvent.sk
12:07:49 encoding ./result/sk_button/View.sk
12:07:49 encoding ./result/sk_button/ViewGroup.sk
12:07:49 encoding ./result/sk_button/Window.sk
12:07:49 encoding ./result/sk_button/WindowManagerGlobal.sk
12:07:49 encoding ./result/sk_button/AbsSpinner.sk
12:07:49 encoding ./result/sk_button/AdapterView.sk
12:07:50 encoding ./result/sk_button/ArrayAdapter.sk
12:07:50 encoding ./result/sk_button/BaseAdapter.sk
12:07:50 encoding ./result/sk_button/Button.sk
12:07:50 encoding ./result/sk_button/CheckBox.sk
12:07:50 encoding ./result/sk_button/CompoundButton.sk
12:07:50 encoding ./result/sk_button/LinearLayout.sk
12:07:50 encoding ./result/sk_button/RadioButton.sk
12:07:50 encoding ./result/sk_button/RadioGroup.sk
12:07:50 encoding ./result/sk_button/RelativeLayout.sk
12:07:50 encoding ./result/sk_button/Spinner.sk
12:07:50 encoding ./result/sk_button/TextView.sk
12:07:50 encoding ./result/sk_button/Toast.sk
12:07:50 encoding ./result/sk_button/ButtonActivity.sk
12:07:50 encoding ./result/sk_button/Main.sk
12:07:50 encoding ./result/sk_button/R.sk
12:07:50 encoding ./result/sk_button/AuxAccessorMap.sk
12:07:55 encoding ./result/sk_button/AuxAccessorUni.sk
12:07:55 encoding ./result/sk_button/AuxObserverMotionEvent.sk
12:07:56 encoding ./result/sk_button/AuxSingleton.sk
12:07:56 encoding ./result/sk_button/sample_sample1.sk
12:07:56 encoding ./result/sk_button/log.sk
12:07:56 encoding ./result/sk_button/sample.sk
12:07:56 sketch running...
[1467356878.0230 - ERROR] [SKETCH] Sketch Not Resolved Error: nullThe sketch could not be resolved.
[1467356878.0280 - DEBUG] [SKETCH] stack trace written to file: ./result/tmp/stacktrace.txt
[1467356878.0281 - DEBUG] Backend solver input file at /home/yh/pasket-master/result/tmp/sk_button/input0.tmp
12:07:58 wrong modelings
I wonder if some configure I did wrong.And I noticed when I run "python -m grammar.gen" it shows a warning which I cannot solve:
java.g:691:9: Decision can match input such as "'0'..'9'{'E', 'e'}{'+', '-'}'0'..'9'{'D', 'F', 'd', 'f'}"
using multiple alternatives: 3, 4
As a result, alternative(s) 4 were disabled for that input
In addition,when I run "./run.py -c codegen" I am faced with a problem that "package sketch.complier.ast.core does not exist" . So I changed the build.xml in this:
<property name="sketch.dir" location="${env.SKETCH_HOME}/.."/> <property name="sketch.jar" value="${sketch.dir}/sketch-1.6.9-noarch.jar"/> ... ... <target name="compile" depends="init" description="compile java sources">     <javac srcdir="${source.dir}" destdir="${build.dir}" classpath="${sketch.jar}"/>  </target>
I directly refer to the jar file.
All I changed are shown above,I am pleased to hope your reply,thanks so much.

Thank you for trying the command I recommended.
Re: grammar.gen
Yes, we always got the same warning whenever we modified the grammar.  So, you can safely ignore that warning.
Re: codegen
Sorry for the confusion, and you're right: if you're using a tar ball, you can swap some pre-defined properties in the build.xml file.  I commented there to explicitly mention the case.
Re: Sketch error
I recently got a new laptop, and installed all the necessary things in a very fresh environment.  I guess you're using sketch-1.6.9 tar ball, whereas I'm using Sketch source directly, which is now v1.7.2beta.  Anyhow, due to the enhanced type checking in Sketch front-end, pasket-generated sketch files got type errors, rather than non-resolve error you got.  In either case, I need to investigate this further.  Thank you for helping me notice this.  Will update anything soon-ish, but not guaranteed, as I'm no longer a grad student.

Thank you for your guiding and I got the Model Synthesis successfully!
Now I tried the Swing Model.I used the NASA jpf sources to compile and I installed the jpf-core,jpf-awt and jpf-symbc successfully in a directory which I create.However , when I executed the command "ant gui" in result dircetory with error outcome.
I check the build.xml with gui target:

there are some questions about it:
1、in ~result/java directory ,it only has com and android directories,and how can I get the directories like java,javax
2、I used the NASA sources and its jpf-awt class directory has no java/util and javax/accessibility directory , if I got the file source wrong?
And if you have some more detailed guiding or infomation about Usage (Model) with Swing?
Thank you a lot!


Update to schemas 1.1.2

See https://github.com/nlpsandbox/nlpsandbox-schemas/issues/213

The Docker image is now pushed to Synapse here.

Done. README will be updated shortly (#63 )


Antisense creation is not possible if non eligable for antisense createion chain present in the selection

Steps to Reproduce

Go to Macro - Snake mode
Load from HELM using paste from clipboard: RNA1{([25moe3],[5A6])([nC6n8A])P}|CHEM1{[4aPEGMal]}|RNA2{R(A)P}$RNA1,CHEM1,2:R2-1:R1$$$V2.0

Select all monomers on the canvas and call context menu

Actual behavior
Create Antisense Strand option is disabled

Expected behavior
Create Antisense Strand option is enabled. Antisense creation is possible

As per requirement:

1.3. If multiple chains are selected and more than one satisfies the previous requirements, the "Create Antisense Strand" option should appear, and if clicked, multiple antisense chains are created.

Versions

Ketcher Version 2.28.0-rc.3 Build at 2024-12-11; 14:32:57
Indigo Version 1.27.0-rc.1.0-g8d656a341-wasm32-wasm-clang-19.0.0
Chrome Version 131.0.6778.109 (Official Build) (64-bit)
Win10

Issue found while testing - https://github.com/epam/ketcher/issues/5678

Not a bug


Update index.js

Does This Count? lol

hahah yes it does count!! Can you resolve conflicts? Sorry I've been merging wildly this morning!


Ensure that passAsProperties also maps to project/system properties.

Starting with the version 1.27 of the gradle plugin the passAsProperties
setting has been split into two settings: passAllAsProjectProperties
and passAllAsSystemProperties.
Now the passAsProperties settings forwards to both passAllAsProjectProperties
and passAllAsSystemProperties to ensure backwards compatibility.
Users can already pass project and system properties explicitly via switches.
Therefore I don't think that any further handling of system/project properties
is needed.

The Gradle plugin handle backwards compatibility, so there should be no need to handle this in Job DSL. See https://github.com/jenkinsci/gradle-plugin/blob/gradle-1.27/src/main/java/hudson/plugins/gradle/Gradle.java#L413-L419
Or am I missing something?

There is some difference: For the Gradle plugin < 1.27, the default was to pass all job properties as system properties. You could select to pass all job properties as Gradle project properties instead by using the passAsProperties flag.
As of 1.27, no job properties are passed to Gradle by default. If we parse an old xml file which still has the passAsProperties flag, then we activate one of passAllAsProjectProperties or passAllAsSystemProperties, depending on the value, for backwards compatibility.
So the job dsl integration may generate the old xml version - so the xml will be migrated correctly. This PR would generate a different logic: activating both passAllAsProjectProperties and passAllAsSystemProperties if passAsProperties is true. I think we should not do that, given that the migration logic in the Gradle plugin itself is different.

Users can already pass project and system properties explicitly via switches.
Therefore I don't think that any further handling of system/project properties
is needed.

Actually, there are now two new fields for passing key/value pairs of system properties and project properties to the Gradle invocation. If you do not want to use those and use switches instead, I think it would be better to deprecate GradleContext.passAsProperties, make it of type Boolean, so null can be used, and only pass in something for the Gradle plugin version >= 1.27 if passAsProperties != null.

The problem I had was that I failed to deactivate overall passing of properties.
With Job DSL 1.66 passAsProperties(false) still leads to passing job parameters as system parameters.


Removes duplicate vulnerabilities from the /images vulnerability responses

What this PR does / why we need it:
It simply removes duplicate CVEs from the /images//vuln/<os|non-os|all> responses
Which issue this PR fixes
#220
Special notes
I haven't added any tests for this, mostly because it's pretty trivial so a regression doesn't matter much but also because I wasn't sure where they'd fit well. If anyone thinks there is somewhere a test would fit nicely, feel free to point me to the file and I'll happily add one :)

Hi @ShahneRodgers , thanks for taking a look at this. However, there are legitimate cases for duplicate vulnerability ids if the package it matches against is different. This PR seems to only key off the vulnerability ID, which could remove useful information from the response. I think more information is needed in the referenced issue before determining what kind of de-duplication is necessary, if any. Specifically, some example duplicates where the records are actually fully identical. Do you have any examples we can discuss?

No worries; I just thought Anchore looked pretty cool and wanted to get some idea of how it worked. I was looking at Alpine:3.9 which returns
{
  "imageDigest": "sha256:bf1684a6e3676389ec861c602e97f27b03f14178e5bc3f70dce198f9f160cce9",
  "vulnerabilities": [
    {
      "feed": "vulnerabilities",
      "feed_group": "alpine:3.9",
      "fix": "1.1.20-r5",
      "nvd_data": [],
      "package": "musl-1.1.20-r4",
      "package_cpe": "None",
      "package_cpe23": "None",
      "package_name": "musl",
      "package_path": "None",
      "package_type": "APKG",
      "package_version": "1.1.20-r4",
      "severity": "High",
      "url": "http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-14697",
      "vendor_data": [],
      "vuln": "CVE-2019-14697"
    },
    {
      "feed": "vulnerabilities",
      "feed_group": "alpine:3.9",
      "fix": "1.1.20-r5",
      "nvd_data": [],
      "package": "musl-utils-1.1.20-r4",
      "package_cpe": "None",
      "package_cpe23": "None",
      "package_name": "musl-utils",
      "package_path": "None",
      "package_type": "APKG",
      "package_version": "1.1.20-r4",
      "severity": "High",
      "url": "http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-14697",
      "vendor_data": [],
      "vuln": "CVE-2019-14697"
    }
  ],
  "vulnerability_type": "os"
}

I hadn't noticed that the package_name is actually different though, so maybe it's not a bug at all. I haven't managed to find any other examples so far, so I'll close this and see if any real duplicates exist.


[storage] a few more metrics and dashboard update

Motivation

backup progress gauges.
separate timers for commits from save_transactions() and the state pruner.
utilize the above and other counters we've been adding and not reflected on the dashboard yet.

set fill=0 in most places - it's much clearer and revealing anomalies way easier.
use irate whenever possible - playing with it I found at 15s interval irate() is not really too jumpy, and in cluster-tests we can really use more sensitive graphs than rate() can provide.

Have you read the Contributing Guidelines on pull requests?
Y
Test Plan
cluster test and import new dashboard.

Related PRs

/land


Deprecate apiserver_longrunning_gauge metric in favor of apiserver_longrunning_requests

What does this PR do?
kube_apiserver metric longrunning_gauge has been deprecated since 1.23 in favor of longrunning_requests.
See PR: https://github.com/kubernetes/kubernetes/pull/103799
More over, the deprecated metric was fully removed in 1.25 https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.25.md#other-cleanup-or-flake-2
This is also a metric that we use in the OOTB dashboard, so this dashboard has an empty widget for all customers running >= 1.25
Motivation

Additional Notes

Review checklist (to be filled by reviewers)

[ ] Feature or bugfix MUST have appropriate tests (unit, integration, e2e)
[ ] PR title must be written as a CHANGELOG entry (see why)
[ ] Files changes must correspond to the primary purpose of the PR as described in the title (small unrelated changes should have their own PR)
[ ] PR must have changelog/ and integration/ labels attached
[ ] If the PR doesn't need to be tested during QA, please add a qa/skip-qa label.

Is this only going to be fixed in Agent version 7.47.0?
Thank you


Example throws : error: dictionary does not contain key "bounds"

Hi, I saw your package and I wanted to use it to illustrate Riemann sums, but I can't get it to work :sob:.
I tried the following example you provide (and others)
#import "@preview/riesketcher:0.2.0": riesketcher

It doesn't work T-T

#riesketcher(
  x => calc.pow(x, 3) + 4,
  method: "left",
  start: -3.1,
  end: 3.5,
  n: 10,
  plot-x-tick-step: 1,
)

and here's what I get :

I'm on typst 0.11.1, on Linux Mint 21.1.
Let me know if I can provide additional info ! :smile:

My bad, I didn't know you needed to wrap up the hole thing in a canvas ^^'

But I still can't compile the examples, and I wonder if I really encountered a bug this time.
Trying to compile the example
#import "@preview/riesketcher:0.2.0": riesketcher
#import "@preview/cetz:0.2.2" : canvas, plot

#canvas({
  riesketcher(
    x => 16 - x * x,
    method: "right",
    end: 6,
    n: 6,
    domain: (-1, auto),
    plot-x-tick-step: 1,
)})

Throws the error
error: dictionary does not contain key "bounds"
   ┌─ @preview/cetz:0.2.2/src/process.typ:18:18
   │
18 │       if drawable.bounds {
   │                   ^^^^^^

help: error occurred in this call of function `element`
   ┌─ @preview/cetz:0.2.2/src/process.typ:63:12
   │
63 │     let r = element(ctx, el)
   │             ^^^^^^^^^^^^^^^^

help: error occurred in this call of function `many`
   ┌─ @preview/cetz:0.2.2/src/canvas.typ:64:33
   │
64 │   let (ctx, bounds, drawables) = process.many(ctx, body)
   │                                  ^^^^^^^^^^^^^^^^^^^^^^^

This was already fixed in v0.2.1

My bad, I'll pay more attention next time


BigSolDB

Probably some overlap with others
https://bigsoldb.cheminfo.space/

added via https://github.com/kjappelbaum/awesome-chemistry-datasets/pull/60


Control Hopper Env, Stand + Hop

Added hopper xml, hopper environment, hopper stand and hop tasks
MS-HopperStand-v1 & MS-HopperHop-v1

Made small changes to the following files
mani_skill/envs/sapien_env.py - added ability to add articulation mount for render only camera - this allows the ppo evaluation videos to be mounted to robot articulation (necessary to view hopper throughout episode and it's movement) - doesn't make any difference for _human_render_cameras if entity_uid in camera config isn't specified - as usual for all previous environments
mani_skill/utils/building/ground.py - added additional kwargs to ground builder to allow rectangular grounds (no longer just square) and ground xy offsets (both necessary for dm_control planar agent ground visual in mujoco planar agent environments style) - no difference for this api if these additional kwargs aren't specified - as usual for all previous environments
index - task documentation

Made large changes to the following files
mani_skill/envs/utils/rewards/common.py - value at margin wasn't implemented, and the sigmoidal reward fns weren't guaranteed to return rewards between 0 and 1! - made changes s.t. it's equivalent to dm_control reward tolerance

These changes make it much easier for implementation of other planar agents (like walker and cheetah), but in my experience, there are issues (like known and documented bugs in mjcf_loader.py and joint tuning difficulty) that make these other environments harder to implement

environment requires further tuning to more resemble dm_control hopper
https://github.com/user-attachments/assets/3412df4c-cb86-401a-ba19-e0209167797b
https://github.com/user-attachments/assets/a791bd42-e712-4ba3-b1e6-61cefa637daa
args to recreate:
Hop
for i in {1..3}; do python ppo.py --exp_name="hopseed${i}" --env_id="MS-HopperHop-v1" --num_envs=2048 --update_epochs=8 --num_minibatches=32   --total_timesteps=20_000_000 --eval_freq=10 --num_eval_steps=100 --num_steps=100 --gamma=0.999 --seed=${i}; done
Stand
for i in {1..3}; do python ppo.py --exp_name="standseed${i}" --env_id="MS-HopperStand-v1" --num_envs=2048 --update_epochs=8 --num_minibatches=32   --total_timesteps=20_000_000 --eval_freq=10 --num_eval_steps=100 --num_steps=100 --gamma=0.99 --seed=${i}; done
HopRGB singleseed
python ppo_rgb.py --exp_name="rgbhopseed1" --env_id="MS-HopperHop-v1" --num_envs=1024 --update_epochs=8 --num_minibatches=32   --total_timesteps=20_000_000 --eval_freq=10 --num_eval_steps=100 --num_steps=100 --gamma=0.99 --seed=1

modified xml file hip range because original hopper robot is bugged - its hip range allows self intersection getting the robot stuck in a certain position, with small probability:

Is the ppo_rgb.py code updated? Seem to run into a few errors due to evaluating 100 steps instead of the max 600.

Is the ppo_rgb.py code updated? Seem to run into a few errors due to evaluating 100 steps instead of the max 600.

yes, I added ppo_rgb 1 line change to get it to work in new commit - there was no check that success exists in final_info dict in ppo_rgb.py, while there is a check in ppo.py
for i in {1..3}; do python ppo_rgb.py --exp_name="pixels_hopseed${i}" --env_id="MS-HopperHop-v1"   --num_envs=256 --update_epochs=8 --num_minibatches=8   --total_timesteps=25_000_000 --num-steps=100 --num_eval_steps=600 --gamma=0.995 --seed=${i}; done

performance is worse without the pressure sensor information

It is hard to tell if it's "worse" since this is a different return scale now? Also i tried running again but the performance after 5M steps looks like this: /home/stao/work/research/maniskill/ManiSkill2/examples/baselines/ppo/runs/rgbhopseed1/videos/8.mp4
Looks like it learns to flip forward to go fast now, instead of the original behavior.

My script:
python ppo_rgb.py --exp_name="rgbhopseed1" --env_id="MS-HopperHop-v1" --num_envs=512 --update_epochs=8 --num_minibatches=32 --total_timesteps=20_000_000 --eval_freq=15 --num_eval_steps=600 --num_steps=100 --gamma=0.99 --seed=1 --no-include-state
``` (merge main for this to work)

This has been my experience with the damping, stiffness, and delta being very important for desired behvior. If the damping especially is too low, then the hopper is unable to upright itself, if it's too high, then it can instead flip to its feet too easily.

can you somehow revert to the settings that got the first video that was working? or was it only working from state inputs

it was only working from state inputs

Arguably though with the flipping it seems like it achieves the same distance if not further than the standard "behavior". I would then say this is more of a result of the reward function. Since its not possible to get the same exact simulation as mujoco, maybe we can just penalize flipping (just penalize the x/y axis rotation velocity should be sufficient, whichever is the axis pointing to the camera).

okay, that sounds good, I will try some experiments with discouraging flipping reward

Maybe behavior will be removed if the agent does not have so much power behind its actions. I can try and experiment as well.

No need to try to experiement, I have found that y axis rotation bounds for reward works well, will run tests and report here later tonight

Maybe behavior will be removed if the agent does not have so much power behind its actions. I can try and experiment as well.

No need to try to experiment, I have found that y axis rotation bounds for reward works well, will run tests and report here later tonight
edit: update - It worked well for a quick check on state obs - to ensure no flipping with large deltas. However, I've tried many, many different things, and unfortunately, any additional reward doesn't fix the problem - in fact, I believe the flipping was originally a local min solution, where hopping would result in greater reward, since the flipping doesn't always activate the standing reward portion of the hopping reward
currently testing out non delta joint pos

what do you mean non delta joint pos? all of them should use that controller

what do you mean non delta joint pos? all of them should use that controller
I tried pd_joint_pos instead of pd_joint_delta_pos is what I meant, didn't work though

I've diagnosed the problem:
Originally, I tried to implement the robot center of mass x velocity, but it didn't work, so I just used the xvelocity of the slider joint holding the hopper in the xz plane (see subtreelinvelx function).
Since the entire robot's center of mass x velocity isn't taken into consideration (and only the torso xvel which is attached to the slider), the robot isn't penalized for flipping in a large circle as long as its torso continues to move in the +x world frame direction
Here is original robot center of mass velocity function, which is slow and doesn't work
    @property  # dm_control mjc function adapted for maniskill
    def subtreelinvelx(self):
        """Returns linvel x component of robot"""
        total_mass = 0.0
        vels = []
        links = self.agent.robot.get_links()
        # skip first three dummy links
        for i in range(3,len(links)):
            vel = links[i].get_linear_velocity() # vel is (b,3)
            mass = links[i].get_mass()[0]
            vels.append(vel * mass.item())
            total_mass += mass.item()
        vels = torch.stack(vels, dim=0) # (#links, b, 3)
        com_vel = vels.sum(dim=0) / total_mass # (b, 3)
        return com_vel[:,0]

it seems like the get_linear_velocity() function doesn't return the center of mass linear velocity of the links?, which is causing the issue.
If calculating the center of mass velocity for an entire robot is non-trivial, this is an issue for hopper + all planar + all humanoid dm_control task implementations, since they com velocity as reward

Lemme get back to you. I think I also just found a bug with the GPU sim linear and angular velocities. CPU sim is correct but on GPU sim they might flipped. Working with @fbxiang now to debug it.

Actually you can use the torso link. It should work. That is the link they are reading the center of mass velocities from. All velocity data is center of mass actually.
Also merge in main before doing more experiments. I just fixed #450 merged in #452 , so the link velocities used or com velocities are wrong lol

**The results have large variance because in both hop and stand tasks, in seed 2, ppo learns to straighten its legs out too early, and doesn't explore tucking them in when hopper is on its front, in order to get up - ppo then gets stuck at this local optimum in seed 2 - This is a a result of lack of exploration in ppo, not a fault of the environment, as shown below
Standing results and command:

for i in {1..3}; do python ppo.py --exp_name="standseed${i}" --env_id="MS-HopperStand-v1" --num_envs=4096 --update_epochs=8 --num_minibatches=32 --total_timesteps=30_000_000 --eval_freq=10 --num_eval_steps=600 --num_steps=100 --gamma=0.99 --ent_coef=0.01 --seed=${i}; done
Hopping results and command:

for i in {1..3}; do python ppo.py --exp_name="hopseed${i}" --env_id="MS-HopperHop-v1" --num_envs=4096 --update_epochs=8 --num_minibatches=32 --total_timesteps=60_000_000 --eval_freq=10 --num_eval_steps=600 --num_steps=100 --gamma=0.99 --ent_coef=0.01 --seed=${i}; done
RGB Hop results and command:

python ppo_rgb.py --exp_name="pixelhop" --env_id="MS-HopperHop-v1" --num_envs=512 --update_epochs=8 --num_minibatches=32 --total_timesteps=40_000_000 --eval_freq=15 --num_eval_steps=600 --num_steps=100 --gamma=0.99 --seed=1 --no-include-state
seed 2 ppo local optimum:
https://github.com/user-attachments/assets/a51c9a28-a76f-4b1a-aa51-92f9f3f486dc
seed 1 and 3 don't have this issue:
seed 1 stand:
https://github.com/user-attachments/assets/56d488ff-e920-4226-bee6-480a3192d90d
seed 3 stand:
https://github.com/user-attachments/assets/63e0408f-e064-4a84-9c89-36e65690f2de
successful hopping now looks like:
(seed 1)
https://github.com/user-attachments/assets/e6675fdc-082c-434d-94fc-567a2c35b24e
and even if the hopper learns to flip, it is a local minimum as it will eventually learn that hopping results in more rewards than flipping, this is because the center of mass of entire robot body reward (flipping doesn't maximize this reward, but hopping does)
seed 3 of hopping command above first learns to flip, then later learns better how to hop to maximize the reward:
https://github.com/user-attachments/assets/a9675146-d10a-4c5c-9274-77cf24fde986


Update deepspeed requirement from <0.6.0 to <0.7.0

🚀 Feature
Re-land https://github.com/PyTorchLightning/pytorch-lightning/pull/13048
Blocked by a failing Lite test:
pytorch_lightning/lite/lite.py:407: in _run_impl
    return run_method(*args, **kwargs)
pytorch_lightning/lite/lite.py:412: in _run_with_strategy_setup
    return run_method(*args, **kwargs)
tests/lite/test_lite.py:402: in run
    model, optimizer = self.setup(model, optimizer)
pytorch_lightning/lite/lite.py:173: in setup
    model, optimizers = self._strategy._setup_model_and_optimizers(model, list(optimizers))
pytorch_lightning/strategies/deepspeed.py:414: in _setup_model_and_optimizers
    self.model, optimizer = self._setup_model_and_optimizer(model, optimizers[0])
pytorch_lightning/strategies/deepspeed.py:426: in _setup_model_and_optimizer
    deepspeed_engine, deepspeed_optimizer, _, _ = deepspeed.initialize(
/usr/local/lib/python3.9/dist-packages/deepspeed/__init__.py:120: in initialize
    engine = DeepSpeedEngine(args=args,
/usr/local/lib/python3.9/dist-packages/deepspeed/runtime/zero/partition_parameters.py:377: in wrapper
    if not hasattr(module, "_ds_child_entered"):
/usr/local/lib/python3.9/dist-packages/deepspeed/runtime/engine.py:432: in __getattr__
    if name in dir(self):
/usr/local/lib/python3.9/dist-packages/torch/nn/modules/module.py:1847: in __dir__
    parameters = list(self._parameters.keys())
/usr/local/lib/python3.9/dist-packages/deepspeed/runtime/engine.py:432: in __getattr__
    if name in dir(self):
/usr/local/lib/python3.9/dist-packages/torch/nn/modules/module.py:1847: in __dir__
    parameters = list(self._parameters.keys())
E   RecursionError: maximum recursion depth exceeded while calling a Python object
!!! Recursion detected (same locals & position)
=========================== short test summary info ============================
FAILED tests/lite/test_lite.py::test_deepspeed_multiple_models - RecursionErr...

If you enjoy Lightning, check out our other projects! ⚡

Metrics: Machine learning metrics for distributed, scalable PyTorch applications.

Lite: enables pure PyTorch users to scale their existing code on any kind of device while retaining full control over their own loops and optimization logic.

Flash: The fastest way to get a Lightning baseline! A collection of tasks for fast prototyping, baselining, fine-tuning, and solving problems with deep learning.

Bolts: Pretrained SOTA Deep Learning models, callbacks, and more for research and production with PyTorch Lightning and PyTorch.

Lightning Transformers: Flexible interface for high-performance research using SOTA Transformers leveraging Pytorch Lightning, Transformers, and Hydra.

cc @carmocca @akihironitta @borda @SeanNaren @awaelchli @rohitgr7

The test fails in the 0.6.0 to 0.6.4 releases with the following error:
        for mw_b, mw_a in zip(state_dict.values(), model.state_dict().values()):
>           assert not torch.allclose(mw_b, mw_a)
E           RuntimeError: The size of tensor a (0) must match the size of tensor b (32) at non-singleton dimension 1

lite/test_lite.py:420: RuntimeError

And in 0.6.5+ with the error reported in the top post. I will try git blaming deepspeed to see if I can find the patch that changed the error

Git bisect points to https://github.com/microsoft/DeepSpeed/pull/1915/files

I opened https://github.com/microsoft/DeepSpeed/issues/2139 for the recursion error in 0.6.5.


Claim in the spa<n

Hello
We can claim the spawn and the warzone

See https://rtdx.aericio.net/en/latest/plugins/piggyfactions/docs/quickstart.html#setting-up-safezones-warzones


[move] Turn down the heat on Cost To Exist

There's a corner case in Cost To Exist, which is creating more burn than is needed for the network to be healthy. It erred on the side of aggressive. That's not in the spirit of the community.
TL;DR
The network is stuck without being able to return to the precious steady-state of 80+ validators on network (see pull request Jail Refactor which addresses this). Since we haven’t reached scale, the Cost To Exist is burning greater balance than expected due to the implementation. Also the time period for abeyance expired, and it was too short for people to think through and plan. This proposal turns down the heat.
Background:
Cost To Exist for validators is intended to be a long multi-year cost for inactivity validators which had an early advantage in joining the network. Instead due to implementation details and the network being at < 40 validators for one month, the burns on inactive accounts are significant.
Plus the time period for proposal activation was too short. The period of abeyance for the proposal to go into effect (at epoch 185), was too short given the multiple constraints in communicating (given all events in the industry of late). There are active contributors only now learning about the proposal.
Last, it is not critical that this proposal should to be activated immediately for it to have its desired effect of coaxing inactive validators to participate, or gradually transfer coins out of their validator account.
Changes in this PR:

Only implement at steady state. The Cost To Exist, will only be implemented at steady state: when 90+ validators out of 100 successfully operating in an epoch.

The cost is reduced. By placing the threshold at 90 validators the cost is exponentially smaller. The maximum cost would be 2,000 GAS per epoch, and goes down to 0 if there are 100 validators on the network.

Escape Hatch. There will be abundant time to evaluate and plan. This proposal can be rejected with an upgrade before epoch 290. Ninety days will be sufficient time for comment, and a broad consensus should be expected at that point.

I agree with pushing back the date for the Cost to exist to exist. The implementation of the 90+ validators is not present yet.

I think this is fine, but as it stands right now, nodes cannot remove themselves from the validator set, which is what I see is the real problem with the automated burning of funds for inactive validators.

LGTM

LGTM

LGTM

LGTM

Lgtm


Crashes on SessionDelegate.remove() when canceling Download Task

Check List
Thanks for considering to open an issue. Before you submit your issue, please confirm these boxes are checked.

[x] I have read the wiki page and cheat sheet, but there is no information I need.
[x] I have searched in existing issues, but did not find a same one.
[x] I want to report a problem instead of asking a question. It'd better to use kingfisher tag in Stack Overflow to ask a question.

Issue Description
What
We are using Kingfisher to download images for our iOS app and I have encountered many crashes when trying to cancel a download task
EXC_BAD_ACCESS KERN_INVALID_ADDRESS 0x0000000000000020
Crashed: com.apple.main-thread
0  CFNetwork                      0x18436b058 HTTPParser::HTTPParser(__CFAllocator const*, HTTPParserClient*, HTTPParser*) + 44
1  CFNetwork                      0x18424b27c HTTPMessage::commonInitialization(unsigned char, HTTPMessage const*) + 212
2  CFNetwork                      0x18424b27c HTTPMessage::commonInitialization(unsigned char, HTTPMessage const*) + 212
3  CFNetwork                      0x18424b41c HTTPMessage::HTTPMessage(HTTPMessage const*) + 112
4  CFNetwork                      0x18436c1dc HTTPRequestMessage::HTTPRequestMessage(HTTPRequestMessage const*) + 36
5  CFNetwork                      0x18422a74c HTTPRequest::HTTPRequest(HTTPRequest const*) + 24
6  CFNetwork                      0x18422ce8c URLRequest::initialize(URLRequest const*, unsigned char) + 656
7  CFNetwork                      0x1841e2bd8 _createRequestCopy(__CFAllocator const*, _CFURLRequest const*, unsigned char) + 152
8  CFNetwork                      0x184355de0 -[NSURLRequest mutableCopyWithZone:] + 44
9  libswiftFoundation.dylib       0x105a95a0c URLRequest.init(_bridged:) + 77948
10 libswiftFoundation.dylib       0x105a99344 static URLRequest._unconditionallyBridgeFromObjectiveC(_:) + 92596
11 Kingfisher                     0x102fb5788 SessionDelegate.remove(_:) + 93 (SessionDelegate.swift:93)
12 Kingfisher                     0x102fb566c closure #1 in SessionDelegate.add(_:url:callback:) + 1000 (<compiler-generated>:1000)
13 Kingfisher                     0x102fb9eb0 partial apply for thunk for @escaping @callee_guaranteed (@guaranteed SessionDelegate, @unowned Int, @in_guaranteed SessionDataTask.TaskCallback) -> () + 84 (<compiler-generated>:84)
14 Kingfisher                     0x102f46734 specialized closure #1 in Delegate.delegate<A>(on:block:) + 96 (<compiler-generated>:96)
15 Kingfisher                     0x102fb9f34 partial apply for specialized closure #1 in Delegate.delegate<A>(on:block:) + 28 (<compiler-generated>:28)
16 Kingfisher                     0x102fb14d0 SessionDataTask.cancel(token:) + 356 (<compiler-generated>:356)
17 Kingfisher                     0x102f6de18 DownloadTask.cancel() + 74 (ImageDownloader.swift:74)
18 MyApp                          0x1011e55f4 UIImageView.cancel() +<PHONE_NUMBER> (<compiler-generated>:4310799860)
...
...

For example we call this method when reusing cells in a tableview, where we cancel requests for cells that are not longer visible. We have created an extension on UIImageView to define convenience methods like cancel()
What could be the reason for this crash? The download task in our code is an optional, so even when the object is not there anymore, calling image?.downloadTask.cancel() should not create crashes, should not even trigger those methods.
Here is a Sample code of what we do:
func fetchImage(fromURL url: URL, completion: @escaping (UIImage?, NSError?) -> Void) -> MyAppDownloadTask {

	let resource = ImageResource(downloadURL: url)
	let options: KingfisherOptionsInfo = [.targetCache(cache), .downloader(downloader)]

	let task = KingfisherManager.shared.retrieveImage(with: resource, options: options, progressBlock: nil) { result in

	switch result {
		case .success(let value):
			completion(value.image, nil)
		case .failure(let error):
			completion(nil, error.toNSError())
	}
        }

	return MyAppDownloadTask(task: task)
}

private class MyAppDownloadTask {

	let task: DownloadTask? // KingFisher task

	init(task: DownloadTask?) {
		self.task = task
	}

	func cancel() {
		task?.cancel()
	}
}

We call the cancel method when reusing cells in our collection view:
override func prepareForReuse() {
	super.prepareForReuse()
	downloadTask.cancel()
}

Reproduce
I cannot share the URL that we call, but we've notice that it happens most of the time when the network request have been canceled from the server.
Info from New Relic:
networkErrorCode: -999
connectionType: LTE
networkError: cancelled
requestMethod: GET
typeOfImage: PNG

Other Comments
The UI updates (assign of UIImageViews) and the cancelation of request when are happening are being done in the Main Thread.

We are also getting the same issue and are using version "5.14.1" via carthage.
Crashed: com.apple.main-thread 0  CFNetwork                      0x199af5434 _CFStreamErrorFromCFError + 3360 1  CFNetwork                      0x199af5630 _CFStreamErrorFromCFError + 3868 2  CFNetwork                      0x199bfe8fc _CFHTTPServerResponseEnqueue + 55568 3  CFNetwork                      0x199adf2ec _CFURLConnectionCopyTimingData + 7676 4  CFNetwork                      0x199a26b70 (Missing) 5  libswiftFoundation.dylib       0x1cd6cb688 $s10Foundation10URLRequestV36_unconditionallyBridgeFromObjectiveCyACSo12NSURLRequestCSgFZ + 68 6  Kingfisher                     0x104fbde5c SessionDelegate.remove(_:) + 93 (SessionDelegate.swift:93) 7  Kingfisher                     0x104fbdd40 closure #1 in SessionDelegate.add(_:url:callback:) + 75 (SessionDelegate.swift:75) 8  Kingfisher                     0x104fc2564 partial apply for thunk for @escaping @callee_guaranteed (@guaranteed SessionDelegate, @unowned Int, @in_guaranteed SessionDataTask.TaskCallback) -> () +<PHONE_NUMBER> (<compiler-generated>:4318127460) 9  Kingfisher                     0x104f6075c specialized closure #1 in Delegate.delegate<A>(on:block:) + 39 (Delegate.swift:39) 10 Kingfisher                     0x104fc25e8 partial apply for specialized closure #1 in Delegate.delegate<A>(on:block:) +<PHONE_NUMBER> (<compiler-generated>:4318127592) 11 Kingfisher                     0x104fc7684 SessionDataTask.cancel(token:) +<PHONE_NUMBER> (SessionDataTask.swift:4318148228) 12 Kingfisher                     0x104fb4b88 KingfisherWrapper<A>.cancelDownloadTask() +<PHONE_NUMBER> (ImageView+Kingfisher.swift:4318071688) 13 RecoverCoreUI                  0x1049e2cfc MessageCell.prepareForReuse() + 202 (MessageCell.swift:202)

We are using 5.14.0 via CocoaPods

Tried to fix in 5.15.5. If anyone can confirm it.

We encountered this same crash when cancelling a bunch of image requests. Our stack trace looks like:
Thread 0 Crashed:
0   CFNetwork                            0x00000001b74573d8 HTTPMessage::commonInitialization(unsigned char, HTTPMessage const*) + 436 (HTTPParser.cpp:64)
1   CFNetwork                            0x00000001b74573bc HTTPMessage::commonInitialization(unsigned char, HTTPMessage const*) + 408 (CFObject.h:189)
2   CFNetwork                            0x00000001b74575b0 HTTPMessage::HTTPMessage(HTTPMessage const*) + 108 (HTTPMessage.cpp:35)
3   CFNetwork                            0x00000001b7560440 HTTPRequestMessage::HTTPRequestMessage(HTTPRequestMessage const*) + 36 (HTTPRequestMessage.cpp:10)
4   CFNetwork                            0x00000001b7442610 URLRequest::initialize(URLRequest const*, unsigned char) + 560 (HTTPRequest.cpp:54)
5   CFNetwork                            0x00000001b7392eb8 -[NSURLRequest mutableCopyWithZone:] + 80 (NSURLRequest.mm:534)
6   libswiftFoundation.dylib             0x00000001cac731d8 static URLRequest._unconditionallyBridgeFromObjectiveC(_:) + 64 (URLRequest.swift:45)
7   HuluAPI                              0x0000000101e3e9cc _hidden#6805_ + 212 (__hidden#6906_:93)
8   HuluAPI                              0x0000000101e3e8b4 _hidden#6804_ + 892 (__hidden#6906_:75)
9   HuluAPI                              0x0000000101e41edc _hidden#6838_ + 60 (__hidden#591_:0)
10  HuluAPI                              0x0000000101e41fa4 _hidden#6839_ + 96 (__hidden#725_:39)
11  HuluAPI                              0x0000000101e3b4a4 __hidden#5908_ + 32 (__hidden#725_:44)
12  HuluAPI                              0x0000000101e3b4a4 _hidden#2197_ + 328 (__hidden#6781_:107)
13  HuluAPI                              0x0000000101df19ac __hidden#2553_ + 8 (__hidden#2554_:73)
14  HuluAPI                              0x0000000101df19ac KingfisherWrapper<A>.cancelDownloadTask() + 32 (__hidden#2331_:249)

From the email chain we had with DTS:
In the backtrace above we see the last two frames end with creating a new HTTPMessage.  What happens in HTTPMessage::commonInitialization is that a new HTTPParser object is created based on the original that is passed into HTTPMessage.  This runs into a race condition with a NSURLSession task when the copy takes place for HTTPParser.  The issue does not exist in iOS 14 / tvOS 14 because a lock was placed around the creation of the new HTTPParser object.

So, how do you workaround this in iOS 13 / tvOS 13 for the use case you have described?  You can still cancel the task, but can’t call properties on the original task until it has completely cancelled.  This is the result of the crash and KERN_INVALID_ADDRESS exception you are getting.  Examples of such properties to stay away from before the request is cancelled would be currentRequest or originalRequest as these will cause this issue.  I would take a look at the Kingfisher library for any instances where task properties are used before the task is completely cancelled.  Look for the usage of currentRequest or originalRequest specifically.

I'm seeing references to URLRequest.originalRequest in SessionDelegate.swift (where it uses originalRequest.url as a tasks lookup key), this looks to be the source of the race condition crash. Since URLRequest is Hashable, can we use the entire request object as the dictionary key, instead of inspecting originalRequest.url?

@danielbyon-hulu Ummm, that is really hidden pitfall. Nice to hear that it is fixed in iOS 14.
We cannot use the request directly as the key. The current implementation is an optimization to prevent multiple downloading from the same URL. Changing an object identifier key (the hash of request, etc) would break this feature.
May I know is it possible for you to build a stable reproducible sample? So we can use that to check whether a fix works or not.
Anyway, I tried to adjust some locks to make sure the accessing of URLRequest.originalRequest properties not happening at the same time as cancel. You can find these changes in #1558
Any feedback is welcome!

Unfortunately, we've only ever seen this crash in NewRelic monitoring, we've never been able to repro locally.


v11.2.2

Current coverage is 97.57%

Merging #230 into master will not affect coverage as of f518ab7

@@            master    #230   diff @@
======================================
  Files           26      26
  Stmts          371     371
  Branches         0       0
  Methods         81      81
======================================
  Hit            362     362
  Partial          0       0
  Missed           9       9

Review entire Coverage Diff as of f518ab7

Powered by Codecov. Updated on successful CI builds.


The application "gonvim.app" cannot be opened.

I finally was able to build gonvim on my machine. However, when I try to launch it I get the following message:

Running from the terminal I can see the error which I don't have any Idea about its solution :(
>> open -a gonvim
LSOpenURLsWithRole() failed for the application /Users/juju/Development/go/src/github.com/akiyosi/gonvim/cmd/gonvim/deploy/darwin/gonvim.app with error -10810.

Hi,
I have never encountered this problem.
I looked into this and seems to have the following permission problem.
Please check your system.
https://discussions.apple.com/thread/7586240

Mac OS 10.14
It does not happen in any of the released versions.

@khalidchawtany Is this still an issue with GoNeovim v0.4.0?

Sorry I gave up compiling the app myself. I always download the from the releases.


refactor: add .component-history into .gitignore

Signed-off-by: pantang<EMAIL_ADDRESS>

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

@GoodMeowing You need to sign the CLA first.

/approve


It takes 27.41 seconds to load this NuGet page...

/cc @skofman1 ref: https://github.com/NuGet/NuGetGallery/issues/4859#issuecomment-337455878
https://www.nuget.org/packages/FAKE/
$(".version-history tr").length
1627

Tool used to check page load performance: https://tools.pingdom.com

Also, my other NuGet issue #4494 is now 77 days old and still a blocking issue to this day. As a reminder, #4494 is a server-side issue (and optionally a client-side issue) that needs to get fixed. I would really appreciate it if someone at Microsoft can put some effort into getting these issues fixed instead of pushing issues to a "future" label.
Again, as a quick refresher, I dare anyone to install the Baseline NuGet package via Command Line:

Thank you

Thanks for reporting! The Gallery should not display thousands of versions on the package display page. An easy fix here would be to display only the latest 100 versions, and displaying a message if there may be more versions.

The actual reason for me to go to the F# FAKE NuGet page was to look for a specific version in the version history that was published 10 months ago. I needed to go back 10 months so I could resolve an build issue and lock down my dependencies to that specific version number. Additionally, I needed to lock down other dependencies around that similar timeframe. In summary, version history is useful.
I think the best idea is not to truncate information from your website but to make it more accessible.

The page is loading slowly even for packages that have ~100 versions. We need to fix this.

There is some pretty bad performance on the most popular packages. I tried the top fix on the search results:

ID
Duration (seconds)
Performance Result

Newtonsoft.Json
6.84
https://tools.pingdom.com/#!/bxyIsD/https://www.nuget.org/packages/Newtonsoft.Json/

NUnit
4.3
https://tools.pingdom.com/#!/vLiaw/https://www.nuget.org/packages/NUnit/

EntityFramework
5.54
https://tools.pingdom.com/#!/nmZ2y/https://www.nuget.org/packages/EntityFramework/

jQuery
7.93
https://tools.pingdom.com/#!/dmuzMt/https://www.nuget.org/packages/jQuery/

HtmlAgilityPack
6.3
https://tools.pingdom.com/#!/bDSeBl/https://www.nuget.org/packages/HtmlAgilityPack/

Something has regressed since the redesign.

It looks like the issue is that we're executing a lot of unnecessary EF queries on cshtml execution.
A CPU profile shows 62% inclusive spent on DisplayPackage.cshtml execution, and just 18.4% on the controller action:

The controller executes 2 queries:

line 415; package = _packageService.FindPackageByIdAndVersion(...)
line 427; packageHistory = ...

The razor view executes (# of versions) + 1 queries, seemingly to populate the Dependencies relation for every package version.
We should probably fix by including the Dependencies relation on the specified (or latest) version, but exclude and don't query Dependencies on all other versions. Still need to test and verify if that will resolve the performance issue.

Validation of FAKE package...

Scenario
Baseline Env
Baseline Result
Env
Result
%Diff

DisplayPackages (1360 versions)
INT-USSC
3217ms
INT0
768ms
-76%

@bchavez You should see faster load times on this page now. I couldn't repro the 27s load time, but saw about 6s load times on average. If I find more optimizations, I'll open new issues to track.

@chenriksson thank you! I did notice recently page times are so much faster now. It's amazing. It's really wonderful to have NuGet working really good once again.
Again, super thanks for working on this and making everything so fast. :+1: :airplane:


Upload FUUN

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

[ ] Filename in the lowercase format
[ ] Image format: png
[ ] Image size: 256 by 256 px

Please, lowercase file name for next upload, for now, added lowercased image file name here: https://github.com/TrustWallet/tokens/commit/03a2689125ba1714b4205d98fbcaf31ffb4db02d
Also, you can use these graphics to say that your ERC20 token fully compatible with Trust Wallet (https://trustwallet.com/logo-assets). Tokens will show up automatically once the user receives them.
Feel free to share this on social media and tag Trust Wallet:

Twitter: @trustwalletapp

Facebook: @trustwalletapp


[Do Not Merge][terra-popup] Migrate docs to terra-framework-docs

Summary

Migrate docs to terra-framework-docs

Closes #
Deployment Link

https://terra-framework-deployed-pr-#.herokuapp.com/
Testing

Additional Details

Thank you for contributing to Terra.
@cerner/terra

closed in https://github.com/cerner/terra-framework/pull/1549


Note about etcd v2-to-v3 upgrade to k8s upgrade

As suggested in https://github.com/projectcalico/cni-plugin/issues/281

LGTM


Added support of kerning to Text

Added support of kerning to Text.
Faced with a strange problem with kerning and phrase "Hello, world", but in other cases kerning has correct behaviour. I think, it is a bug of NSAttributedString.

Hi Andrew,
Thank you for your work! I'll review it and merge in the beginning of the next week.

What do think about make kerning property an optional property?

I was thinking about this, however logically no kerning equals to kerning = 0, so I think zero value is fine.

With and without 0 the kerning has different behaviour (look at link that I've added to my PR).
I do not add Kern attribute to NSAttributedString if kerning is equal 0.

Hi Andrew,
I agree, no matter how it works, logically zero kerning means no kerning and we don't need to fix API to match current implementation. So your workaround is fine. I'm going to merge your patch now. Thank you for your work!