[IN PROGRESS] Address Class

Codecov Report

Merging #13 (fb9d124) into main (19e6dfd) will decrease coverage by 3.81%.
The diff coverage is 73.52%.

@@             Coverage Diff             @@
##              main      #13      +/-   ##
===========================================
- Coverage   100.00%   96.18%   -3.82%
===========================================
  Files            7        8       +1
  Lines          204      236      +32
===========================================
+ Hits           204      227      +23
- Misses           0        9       +9

Impacted Files
Coverage Δ

lib/eth/address.rb
70.96% <70.96%> (ø)

lib/eth.rb
100.00% <100.00%> (ø)

lib/eth/key.rb
100.00% <100.00%> (ø)

spec/eth/key_spec.rb
100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 19e6dfd...fb9d124. Read the comment docs.


Log4 j2 3168 overwritten rotated logs

This fixes an issue in the rolling file appender. The increment counter in the filePattern didn't work if the filenames contained regex characters. In our case it was the '+' character introduced by the timezone in a timestamp (app-20211029T123000+0200.%i.log), that caused filenames to be missed by the getEligibleFiles method.
The fix quotes the parts of the filenames that are not meant to be treated as regex.
I think this is the actual cause for JIRA ticket https://issues.apache.org/jira/browse/LOG4J2-3168 not the compression mentioned in the ticket.
The first commit in this PR has to be reverted before merging, I kept it for now in case there's more work to be done on the PR.
Please let me know what you think.

Because of the 2 bad files I had to manually create a patch and apply it, but this patch has been applied.

Tanks a lot! :)


fix: adjust release workflow

Pull Request type

Please check the type of change your PR introduces:

[ ] Bugfix
[ ] Feature
[ ] Code style update (formatting, renaming)
[ ] Refactoring (no functional changes, no API changes)
[ ] Build-related changes
[ ] Documentation content changes
[ ] Other (please describe):

What is the current behavior?

Issue Number: N/A
What is the new behavior?

Does this introduce a breaking change?

[ ] Yes
[ ] No

Other information

:tada: This PR is included in version 1.0.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Access exception objects?

I am not udnerstanding if maybe this is not implemented?
It would be useful to be able to access the exceptions java raises and all its methods already proxified

That is a good idea, I'll add this...eventually.


Ability to configure / change the loading state in the tab: "Updating..."

Right now, dash-renderer updates the page tab whenever there is a request in flight.
For fast updating callbacks, e.g. dcc.Interval, this becomes pretty annoying: https://community.plot.ly/t/how-to-avoid-updating-html-browser-title-when-using-interval-event/15379
This is one of the few "hardcoded" features in Dash that we should make configurable:

We could make this a config option to turn on/ off or change the text
We could remove it from dash-renderer and attempt to roll it in to loading states. However, loading states are currently configurable only per-component, not across the entire app.

Seems like (1) would be the way to go right now. It could be something like
update_page_title_during_requests = true / false / 'Loading ...'

I propose adding a config store option and adding a config value to dash, either a null value to not change the title or a string Loading .... Which should be the default null or Loading ... ?


simpler cats possible at 32x64

https://github.com/hartsantler/Pytorch-Sketch-RNN
I have forked and modified this script to work with the latest pytorch.  I'm not sure if there is a bug in my fixes, i have made the network much smaller 32x64, and after awhile training, i do not see it converge to cat shapes.  I also tried reducing the number of examples batch into the training, from 70,000 to 64, is that where i am going wrong?
https://youtu.be/doaU6xm0xH8
https://youtu.be/r7o8AGnsANY
https://youtu.be/6F0XcZG1RNg

Hey harsantler,
I have the same issue as you, did you solve it?
Also to run with Cuda I add to do some minor changes..

Hi !
@thomaskerdreux, see issue 3 for  training fix.
@hartsantler, the related bug (a missing transpose) is also present in your code, which might be causing the training failure (the videos you show are similar to what I observe when I leave this bug)
Good luck !

I fixed the issue in question (https://github.com/alexis-jacq/Pytorch-Sketch-RNN/issues/3), now it should work.

@thomaskerdreux
I fixed the transpose bug as pointed out by hassony2.  This has improved results, right away, i can see it making better attempts at drawing a cat.  But looks like 32x64 is too small a network to solve this problem, or maybe i need to let it train longer? But its already at epoch 17,000 and not yet converged to clearly and consistently drawing cat like shapes.
https://youtu.be/iKiWtgWnwQk

I tried a few different options for the hyper parameters, but could not get a 64x64 network to draw as well as i had hoped.  Closing this issue because i'm assuming that it is not possible to get decent results from a 32x64 network, or even a 64x64 net.
https://youtu.be/vMjGkK20pe8


Extra Order By options for Query Limit

i want to be able to download a wide range of maps but only a specific amount
the only options here are low to high and high to low which is not very useful
maybe add an option to randomly select maps in the requested query? like if there's 50 maps but you only want to download 10 then you can just have the query randomly select 10 maps out of that 50

this is a suggestion but i don't know how to label here lol


“Convert all link paths to relative”，be effective only in the current file

now it is effective in all the vault, but sometimes i just want it is effective only in the current file

Secont that!

Added in https://github.com/dy-sh/obsidian-consistent-attachments-and-links/releases/tag/3.2.0


Beef up interpolation tests and interleave function to handle more edge cases

I found a few edge cases, and while working on #59 I also just saw that tests for interleave were a bit lacking, so I felt like adding a few tests there that weren't braking but just for good form as well. I might have some more edge cases in mind as well. This is WIP but will resolve #62 and is related to #52.
It is branched off from #59 at the moment so until that is merged the diff here is a bit overkill.

Added my proposed fix and put limitations in comments as this approach, though in my opinion much cleaner and better, still has some of the same drawbacks and assumptions the old one had, such as not supporting #13 (interpolating properties), interpolating several times to form one complete declaration etc. ${property}: ${value} would for example fail as well. But with a lot if not all of these I don't see any good way to do it without actually parsing the javascript and getting the values <.<. So I guess for now we should just keep assuming what is the most general practices (as I believe we currently are) and maybe then just add a bit more to the readme?

Just remembered another one, it would also fail right now on something along the lines of ```css
display: ${'block;'}
color: red;
aka when the semicolon is inside the interpolated value.

I still believe in what I said above, all these failing test cases I have, should I add them to some test file with "known problems" that we could somehow make not make travis stop us from pushing, or just add them to the README, or a third option?

Coverage decreased (-0.008%) to 99.367% when pulling fa49dc9bce367b662e57e8abdfb37c1e5586583f on emilgoldsmith:fix/#62/one-line-css-interpolations into a4ce9db7c96d7f5d5f03b1897ebb65345cd66e48 on styled-components:master.

Okay I added some quite comprehensive tests. The test file is maybe nearing a size where it should be modularized though?
Also, I kept all the expressions' name properties even though they aren't technically needed with the changes I made in #59 regarding inserting dummyValue but I think it makes the tests more readable than just giving an array of empty object literals.
I just thought of some more edge cases just now regarding declaration blocks so I'll just add a few tests for that and modify the function I use to check if previous declaration was completed a tiny bit and then I think I'm pretty happy with this PR though it should not be merged in until #59 is as it's afollow up to that one.

Coverage decreased (-0.004%) to 99.371% when pulling 400c41cdf36ce0f71e888336bd8dffd0dece75e7 on emilgoldsmith:fix/#62/one-line-css-interpolations into a4ce9db7c96d7f5d5f03b1897ebb65345cd66e48 on styled-components:master.

Coverage decreased (-0.004%) to 99.371% when pulling 870bc5524b913c0526e54d631e6c5b5e2d075eb8 on emilgoldsmith:fix/#62/one-line-css-interpolations into a4ce9db7c96d7f5d5f03b1897ebb65345cd66e48 on styled-components:master.

IMO all is good now here except for waiting for #59 to be merged in before as this PR extends it.

Now there's conflicts, my bad 😢 Don't worry about coveralls we can make it less strict.

Yep no problem, I'll resolve those with a quick rebase, that was expected when I branched off a different branch that I was also changing :).

Yep a plain rebase didn't even have any conflicts, so everything should pass now.
The only thing I'm still curious about is whether we should add something to the README for this one? As in related to my previous comments:

not supporting #13 (interpolating properties), interpolating several times to form one complete declaration etc. ${property}: ${value} would for example fail as well. But with a lot if not all of these I don't see any good way to do it without actually parsing the javascript and getting the values <.<. So I guess for now we should just keep assuming what is the most general practices (as I believe we currently are) and maybe then just add a bit more to the readme?

Just remembered another one, it would also fail right now on something along the lines of

display: ${'block;'}
color: red;
aka when the semicolon is inside the interpolated value.

I still believe in what I said above, all these failing test cases I have, should I add them to some test file with "known problems" that we could somehow make not make travis stop us from pushing, or just add them to the README, or a third option?

I actually also had some more thoughts on how we could possibly handle these edge cases. But I have to get to work now haha ;). I'll open a discussion issue about a possible fix though.

Coverage decreased (-0.008%) to 99.371% when pulling 1ea7f85220d3990aaa8e35ab0f16c3c7f5b7c1b9 on emilgoldsmith:fix/#62/one-line-css-interpolations into 4706de2ea28ca99994ec8c5df309e60a3fd9f459 on styled-components:master.

Thank you!!
I've updated coveralls to be less strict for the future 👌


Update language track description re: babel vs traceur

The user-facing description of the language track still refers to Traceur:
http://help.exercism.io/getting-started-with-ecmascript.html
We should update that.  Where does the helpfile live?

it's in the https://github.com/exercism/docs repo... but I'm not able to deploy it at the moment because I'm getting a node error that makes no sense.

@kytrinyx let me know when it's working.  I can work with @rchavarria on making appropriate edits.  Thanks!

Will do.

@rchavarria it looks to me like it's just a matter of changing the three references to Traceur, along with the hyperlinks.  Am I missing anything?

I'm going to move all of the documentation over to the language-specific repositories today and serve it from the problems api: https://github.com/exercism/exercism.io/pull/2628
I'll be getting rid of the help.exercism.io app altogether.

@matthewmorgan We could wait until @kytrinyx moves the documentation. And yep, you're right, we should update the doc and point to Babel instead of Traceur. It's just a matter of links, and it doesn't affect to exercises code.

I'm going to work on the documentation today, and since it will be in your own repository after that it will be much, much easier to fix!

Fixed in #89


formulae_dependents: fix linkage test issues

We see two kinds of spurious linkage test issues in CI:

tests failing when they should succeed (e.g. Homebrew/homebrew-core#85367)
tests succeeding when they should fail (#611).

This should fix both problems by either uninstalling dependents that
aren't necessarily around at runtime, or linking dependencies that are.
Closes #611.

Actually, this probably doesn't close #611. Will need to think about this some more.


rememberCursorPositionProvider() doesn't register mouse position when clicking while moving and previous popup still open

When using the androidx.compose.ui.window.Popup() we provide popupPositionProvider = rememberCursorPositionProvider(). This function is what determines the mouse location to open the popup. It works perfectly fine when you perform the right-click without moving the mouse. However, sometimes your mouse moves a little when you right-click (this should be expected as normal as our hands can shake a tiny bit). This is also fine, if you right-click (and move) when there is NO popup shown.
However, if a popup is currently visible, and you right-click-and-move (outside the first popup) to trigger another popup, then the popupPositionProvider does NOT get updated with the new mouse location. I suspect this is either because
a) inside rememberCursorPositionProvider() we remember { } the IntOffset of the mouse, or
b) when when we right-click-and-move outside the popup, the onDismissRequest() function does not get called.
Can we get some help on getting this fixed? Thanks, guys!

Please check the following ticket on YouTrack for follow-ups to this issue. GitHub issues will be closed in the coming weeks.


[Licensing/Copyrights/SPDX] Consider FSFE's REUSE spec compliance

Problem statement(s)

There are files in Matter that have proprietary license/copy headers, e.g src/lwip/cyw30739/arch/cc.h
Repo is under Apache-2.0 only yet, some files are BSD-3-Clause, some are MIT, there are few GPLv2 with exception (that might need to be verified if it applies here?)
Many, many files have no header at all
Some smaller copyright attributions that were not changed from boilerplate like "the original author or authors", or "Project Matter Authors"

Potential solution:
[REUSE specification] defines a standardized method for declaring copyright and licensing for software projects. The goal of the specification is to have unambiguous, human- and machine-readable copyright and licensing information for each individual file in a project. Ideally this information is embedded into every file, so that the information is preserved when the file is copied and reused by third parties.
REUSE Spec
Why would it be useful?

It is not straightforward to check if license headers are used consistently (I found that many license/copyright file headers had small whitespace related differences)
It is easy to change a license text to make it a different license while having it look like the original license confusing license checkers
Licensing for other materials than source code (e.g. docs, readme, etc)
Ability to clearly communicate mixed licensing of a file (e.g. Apache-2.0 OR MITl; GPL-3.0 AND MIT;)
Having License and Copyright information machine readable makes it possible to have easy check in CI if new files are complying with the rule

Some notable users of REUSE spec are: Linux Kernel, KDE, QT
Potential issues:

It would be good if we could propagate those changes to third_party repositories wherever possible but we can run the check before pulling submodules
REUSE requires for ALL files to be licensed (including test data, documentation, etc). What license those should have? I think the Zigbee Alliance should decide as I understand the CLA?

I'm currently grouping files by potential issues and resolving easy cases myself, pinged you also on Slack. Thanks!


Fix make_release workflow to properly handle alpha and beta releases

References and relevant issues
Description
For the current implementation, prerelease is set to true only if rc is in reference. So it set to true if ref is refs/tags/v0.5.0rc1 and not set if refs/tags/v0.5.0 or refs/tags/v0.5.0a1
This PR fixes it to properly handle refs like refs/tags/v0.5.0a1 or refs/tags/v0.5.0b10

I have tested similar changes here https://github.com/4DNucleome/PartSeg/blob/develop/.github/workflows/make_release.yml


More verbose debug variable values

Hi,
I am working on a debugger on top of this package.
For that I have extended the behaviour of newVar to have better visibility of the actual values.
func (a *Adapter) newVar(name string, rv reflect.Value) *dap.Variable

(this is screenshot from Visual Studio Code working with my debugger.
The changes are here:
https://github.com/emicklei/yaegi-debug-adapter/blob/89cbdd701673086c0200b5aa7e7cf431b376a122/variables.go#L86
I am happy submit a PR but would like to have feedback on the idea and approach before doing so.

At a glance from the diff https://github.com/traefik-contrib/yaegi-debug-adapter/compare/main...emicklei:yaegi-debug-adapter:feature/more-newVar-cases it looks like a reasonable approach. Please open a PR and I'll look at it more in-depth there.


Fix syntax error in README code example [skip ci]

even? and odd? as variable names cause syntax errors.
Replace with even and odd.
Thanks for the interesting gem!

Thanks for the fix!


Runtime problem "JIT compilation failed..." when providing services from (3rd party) npm libraries (e.g. transloco)

I got an error in the browser when serving my project using "rsbuild dev". I reproduced the issue in the apps/rsbuild/csr/css  sample project by integrating transloco as described here (https://jsverse.github.io/transloco/docs/getting-started/installation).
One thing to mention is, that I installed transloco with "--legacy-peer-deps" due to a conflicting peer dependency with "chokidar" library.
The error message is:
JIT compilation failed for injectable class DefaultTranspiler {
    config = (0,_angular_core__WEBPACK_IMPORTED_MODULE_1__.inject)(TRANSLOCO_CONFIG, { optional: true }) ?? defaultConfig;
    get interpolationMatcher() {
        return resolveM…

Uncaught Error: The injectable 'DefaultTranspiler' needs to be compiled using the JIT compiler, but '@angular/compiler' is not available.

The injectable is part of a library that has been partially compiled.
However, the Angular Linker has not processed the library such that JIT compilation is used as fallback.

Ideally, the library is processed using the Angular Linker to become fully AOT compiled.
Alternatively, the JIT compiler should be loaded by bootstrapping using '@angular/platform-browser-dynamic' or '@angular/platform-server',
or manually provide the compiler with 'import "@angular/compiler";' before bootstrapping.
    at getCompilerFacade (core.mjs:2524:1)
    at Module.ɵɵngDeclareFactory (core.mjs:38883:1)
    at <static_initializer> (jsverse-transloco.mjs:281:1)
    at<EMAIL_ADDRESS>(jsverse-transloco.mjs:189:1)
    at __webpack_require__ (main.ts:5:1)
    at fn (main.ts:5:1)
    at ./src/app/app.config.ts (app.component.ts:11:1)
    at __webpack_require__ (main.ts:5:1)
    at fn (main.ts:5:1)
    at ./src/main.ts (transloco-loader.ts:5:1)

Oh I know why this is.
I was hoping this wouldn’t be the case but I was trying to limit the amount of pre-processing that needed to be done during build on npm packages to just @angular/* packages.
It looks like I’ll need to expand that to either find packages using @angular or just all npm packages.
I’d rather not do the latter, as it will increase build time.

Yes, this version works! Thanks a lot! Btw, you're doing a really nice job!

Maybe there is some other issue with JSNLog (https://jsnlog.com/). When trying to access the JSNLog object, I get the following error message:
ERROR TypeError: (0 , jsnlog__WEBPACK_IMPORTED_MODULE_1__.JL) is not a function
    at new AppComponent (app.component.ts:17:1)
    at NodeInjectorFactory.AppComponent_Factory [as factory] (app.component.ts:12:1)
    at getNodeInjectable (core.mjs:5287:1)
    at createRootComponent (core.mjs:17524:1)
    at ComponentFactory.create (core.mjs:17380:1)
    at ApplicationRef.bootstrap (core.mjs:22748:1)
    at core.mjs:34735:1
    at _ZoneDelegate.invoke (zone.js:369:1)
    at Object.onInvoke (core.mjs:6495:1)
    at _ZoneDelegate.invoke (zone.js:368:1)

I added the code to the app.component.ts like this. Any ideas?
import { Component } from '@angular/core';
import { RouterModule } from '@angular/router';
import { NxWelcomeComponent } from './nx-welcome.component';
import { JL } from "jsnlog";

@Component({
  imports: [NxWelcomeComponent, RouterModule],
  selector: 'app-root',
  templateUrl: './app.component.html',
  styleUrl: './app.component.css',
})
export class AppComponent {
  title = 'rsbuild-csr-css';

  constructor() {
    JL().info('>>>App started');
  }
}

I'm going to open a different issue for jsnlog.
It's behaving extremely strangely and could be related to how it is packaged.


[DML/IE] Re-implement and optimize clamp

Need to re-implement clamp with dml::Clip when the min and max parameters are both float values as following description:
struct DML_ELEMENT_WISE_CLIP_OPERATOR_DESC {
  const DML_TENSOR_DESC *InputTensor;
  const DML_TENSOR_DESC *OutputTensor;
  const DML_SCALE_BIAS  *ScaleBias;
  FLOAT                 Min;
  FLOAT                 Max;
};

Also need to refactor Clamp implementation on IE.

Close it by https://github.com/webmachinelearning/webnn-native/pull/121


Switch from dart-sys-fork to dart-sys

Changes
Fixes #2434.
Checklist

[x] An issue to be fixed by this PR is listed above.
[x] New tests are added to ensure new features are working. Please refer to this page to see how to add a test.
[ ] ./frb_internal precommit --mode slow (or fast) is run (it internal runs code generator, does auto formatting, etc).
[x] If this PR adds/changes features, documentations (in the ./website folder) are updated.
[ ] CI is passing. Please refer to this page to see how to solve a failed CI.

Remark for PR creator

./frb_internal --help shows utilities for development.
If fzyzcjy does not reply for a few days, maybe he just did not see it, so please ping him.

Looks like the next subpart needs update as well...

Well... Looks like we need to change other things to match the update. If you would like to do it feel free to do so! Alternatively, just revert the generate-internal-blahblah code and merge it first, and I will fix those updates.

@fzyzcjy ,

Well... Looks like we need to change other things to match the update. If you would like to do it feel free to do so! Alternatively, just revert the generate-internal-blahblah code and merge it first, and I will fix those updates.

I fixed that problem, so we can merge, I think.

Great job!

@all-contributors please add @evdokimovs for code

Probably within a day :)

Now!


FE 202 Organization Details and Edit

Tech notes
Organization Detail scene and the Edit modal.  These are both very dry because the only relevant  data available is the name.
The storybook component actually makes the gql query and updates the name.  Is that ok?

  Click on edit to change the name.

@CarsonF ready for re-review


Guidance on certificate

I've been successful at running the samples and I'm trying to setup my own identity provider in node.js/express/express-saml2 and I'm testing it against a locally run instance of gitlab community. Trying naively, it almost works as from gitlab I can go to my identity provider, login and post the response. But then gitlab reply with:
Could not authenticate you from SAML because "Found an unexpected number of signature element. saml response rejected".
So most likely I'm doing something wrong but not sure what.
One thing that I could not find is a clear explanation of all the certificates and keys. It does not say what they are, how to generate them properly, if they are required. It would be nice to have a little bit of guidance here.
For example are the X509Certificate in the metadata_idp1.xml file related to the key we can find in key/idp?

@manu-silicon Thank you so much for your valuable feedback.
The signing/encryption key can be generated using the following commands.
$ openssl genrsa -passout pass:foobar -out encryptKey.pem 4096
$ openssl req -new -x509 -key encryptKey.pem -out encryptionCert.cer -days 3650

If you are using 3rd party identity provider like OneLogin and Okta, the IdP metadata is allowed for you to download instead of creating manually. In this module, a helper function allows you to export the metadata after you have constructed the entity (See here).
For the above error, Could not authenticate you from SAML because "Found an unexpected number of signature element. saml response rejected"..  It really depends on how Gitlab (SP) handles the response sent from the express-saml2's IdP, it would be good if you can provide the SAML response as well.
I thought Gitlab is using omniauth-saml (https://about.gitlab.com/2016/03/30/feature-highlight-saml/), and this one relies on ruby-saml maintained by OneLogin, so the error could be thrown from the following lines of code.
https://github.com/onelogin/ruby-saml/blob/master/lib/onelogin/ruby-saml/response.rb#L548-L550
I think the signature element might not be constructed well fit to the Gitlab's context, are you using the default SAML response or using a customize template in express-saml2 ?

Thanks for getting back to me and the command line for generating the signing and encryption key. I've taken care of that.
I'm trying to build my own IdP so I'm testing the gitlab as a service provider and I'm following the gitlab instructions to enable SAML.
Gitlab provides me with the following metadata file:
<?xml version='1.0' encoding='UTF-8'?>
<md:EntityDescriptor ID='_a1eaa7d4-4ea7-4a8d-a36a-fa280e7a4f50' entityID='https://gitlab' xmlns:md='urn:oasis:names:tc:SAML:2.0:metadata' xmlns:saml='urn:oasis:names:tc:SAML:2.0:assertion'>
    <md:SPSSODescriptor AuthnRequestsSigned='false' WantAssertionsSigned='false' protocolSupportEnumeration='urn:oasis:names:tc:SAML:2.0:protocol'>
        <md:NameIDFormat>urn:oasis:names:tc:SAML:2.0:nameid-format:persistent</md:NameIDFormat>
        <md:AssertionConsumerService Binding='urn:oasis:names:tc:SAML:2.0:bindings:HTTP-POST' Location='https://gitlab/users/auth/saml/callback' index='0' isDefault='true'/>
        <md:AttributeConsumingService index='1' isDefault='true'><md:ServiceName xml:lang='en'>Required attributes</md:ServiceName>
            <md:RequestedAttribute FriendlyName='Email address' Name='email' NameFormat='urn:oasis:names:tc:SAML:2.0:attrname-format:basic'/>
            <md:RequestedAttribute FriendlyName='Full name' Name='name' NameFormat='urn:oasis:names:tc:SAML:2.0:attrname-format:basic'/>
            <md:RequestedAttribute FriendlyName='Given name' Name='first_name' NameFormat='urn:oasis:names:tc:SAML:2.0:attrname-format:basic'/>
            <md:RequestedAttribute FriendlyName='Family name' Name='last_name' NameFormat='urn:oasis:names:tc:SAML:2.0:attrname-format:basic'/>
        </md:AttributeConsumingService>
    </md:SPSSODescriptor>
</md:EntityDescriptor>

and my Idp metadata is:
<?xml version="1.0"?>
<md:EntityDescriptor xmlns:md="urn:oasis:names:tc:SAML:2.0:metadata" entityID="https://idp/sso/metadata/gitlab" ID="pfxbcb19ef2-5fa9-d581-a80e-d6ba2be0e2de">
    <md:IDPSSODescriptor WantAuthnRequestsSigned="false" protocolSupportEnumeration="urn:oasis:names:tc:SAML:2.0:protocol">
        <md:KeyDescriptor use="signing">
            <ds:KeyInfo Id="SM4cee9750ee17bb2f60ddbe556a2798218f7a5bbed"
             xmlns:ds="http://www.w3.org/2000/09/xmldsig#">
                <ds:X509Data>
                    <ds:X509Certificate>...</ds:X509Certificate>
                </ds:X509Data>
            </ds:KeyInfo>
        </md:KeyDescriptor>
        <md:KeyDescriptor use="encryption">
            <ds:KeyInfo xmlns:ds="http://www.w3.org/2000/09/xmldsig#">
                <ds:X509Data>
                    <ds:X509Certificate>...</ds:X509Certificate>
                </ds:X509Data>
            </ds:KeyInfo>
        </md:KeyDescriptor>
        <md:NameIDFormat>urn:oasis:names:tc:SAML:2.0:nameid-format:persistent</md:NameIDFormat>
        <md:NameIDFormat>urn:oasis:names:tc:SAML:1.1:nameid-format:emailAddress</md:NameIDFormat>
        <md:NameIDFormat>urn:oasis:names:tc:SAML:2.0:nameid-format:transient</md:NameIDFormat>
        <md:NameIDFormat>urn:oasis:names:tc:SAML:2.0:nameid-format:entity</md:NameIDFormat>
        <md:NameIDFormat>urn:oasis:names:tc:SAML:1.1:nameid-format:unspecified</md:NameIDFormat>
        <md:NameIDFormat>urn:oasis:names:tc:SAML:2.0:nameid-format:kerberos</md:NameIDFormat>
        <md:NameIDFormat>urn:oasis:names:tc:SAML:1.1:nameid-format:WindowsDomainQualifiedName</md:NameIDFormat>
        <md:NameIDFormat>urn:oasis:names:tc:SAML:1.1:nameid-format:X509SubjectName</md:NameIDFormat>
        <md:SingleSignOnService Binding="urn:oasis:names:tc:SAML:2.0:bindings:HTTP-Redirect" Location="https://idp/sso/login/gitlab"/>
        <md:SingleSignOnService Binding="urn:oasis:names:tc:SAML:2.0:bindings:HTTP-POST" Location="https://idp/sso/login/gitlab"/>
        <md:SingleLogoutService Binding="urn:oasis:names:tc:SAML:2.0:bindings:HTTP-Redirect" Location="https://idp/sso/logout/gitlab"/>
        <md:SingleLogoutService Binding="urn:oasis:names:tc:SAML:2.0:bindings:HTTP-POST" Location="https://idp/sso/logout/gitlab"/>
    </md:IDPSSODescriptor>
</md:EntityDescriptor>

I've captured the SAML request and response using the SAML Chrome panel plugin and here is what I have:
<samlp:AuthnRequest AssertionConsumerServiceURL='https://gitlab/users/auth/saml/callback'
    Destination='https://idp/sso/login/gitlab' ID='_01c63c8d-832c-4371-9d92-f5a71d1f9256'
    IssueInstant='2017-05-10T04:42:05Z' Version='2.0' xmlns:saml='urn:oasis:names:tc:SAML:2.0:assertion'
    xmlns:samlp='urn:oasis:names:tc:SAML:2.0:protocol'>
    <saml:Issuer>https://gitlab</saml:Issuer><samlp:NameIDPolicy AllowCreate='true' Format='urn:oasis:names:tc:SAML:2.0:nameid-format:persistent'/></samlp:AuthnRequest>

And here is the response:
<samlp:Response Destination="https://gitlab/users/auth/saml/callback"
    ID="5745299a-d46a-4a1f-8bfa-e13112e2648a" InResponseTo="_01c63c8d-832c-4371-9d92-f5a71d1f9256"
    IssueInstant="2017-05-10T04:42:05.453Z" Version="2.0"
    xmlns:saml="urn:oasis:names:tc:SAML:2.0:assertion"
    xmlns:samlp="urn:oasis:names:tc:SAML:2.0:protocol">
    <saml:Issuer>https://idp/sso/metadata/gitlab</saml:Issuer>
    <samlp:Status><samlp:StatusCode Value="urn:oasis:names:tc:SAML:2.0:status:Success"/></samlp:Status>
    <saml:Assertion ID="6936418b-993a-417c-81ff-c03034fb7fa1" IssueInstant="2017-05-10T04:42:05.453Z"
        Version="2.0" xmlns:saml="urn:oasis:names:tc:SAML:2.0:assertion">
        <saml:Issuer>https://idp/sso/metadata/gitlab</saml:Issuer>
        <saml:Subject>
            <saml:NameID Format="urn:oasis:names:tc:SAML:1.1:nameid-format:emailAddress">emmanuel.stapf@xenko.com</saml:NameID>
            <saml:SubjectConfirmation Method="urn:oasis:names:tc:SAML:2.0:cm:bearer"><saml:SubjectConfirmationData InResponseTo="_01c63c8d-832c-4371-9d92-f5a71d1f9256"
                NotOnOrAfter="2017-05-10T04:47:05.453Z" Recipient="https://gitlab"/></saml:SubjectConfirmation>
        </saml:Subject>
        <saml:Conditions NotBefore="2017-05-10T04:42:05.453Z" NotOnOrAfter="2017-05-10T04:47:05.453Z">
            <saml:AudienceRestriction>
                <saml:Audience>https://gitlab</saml:Audience>
            </saml:AudienceRestriction>
        </saml:Conditions>
    </saml:Assertion>
</samlp:Response>

You are right that the line of code you selected is the culprit. In my case I have 0 signature_nodes and true for signature_elements.empty? I tried to set WantAuthnRequestsSigned to true in my IdP but then the error from ruby-saml occurs at line https://github.com/onelogin/ruby-saml/blob/master/lib/onelogin/ruby-saml/response.rb#L534
Thanks for any insight you can provide!

Apparently the error is not caused by he request but response, so changing the wantAuthnRequestSigned is not workable.
In our IdP, signing of assertion is determined by the pairwise SP setting. WantAssertionsSigned should be defined in the SP metadata or set wantAssertionSigned as true in the SP setting object when you construct your SP.
For login SP, the user information for example the email, first name, last name ...etc, will be included in the assertion, and sign the assertion.

@manu-silicon However, the SP metadata states that AuthnRequestsSigned='false' and WantAssertionsSigned='false', so the unsigned request is fine and the response is supposed to be not signed as well.
It's interesting, so I am considering to open a new ticket for Gitlab example.

Is there anything I can do to help in this case?

@manu-silicon Would you like to share how you setup your Gitlab server, following this tutorial ?

I've followed this tutorial to the letter. The only difference is that my idp_cert_fingerprint. For that I put the one I got from the signing key.

To be complete, here is what I have:
### OmniAuth Settings
###! Docs: https://docs.gitlab.com/ce/integration/omniauth.html
gitlab_rails['omniauth_enabled'] = true
gitlab_rails['omniauth_allow_single_sign_on'] = ['saml']
gitlab_rails['omniauth_block_auto_created_users'] = false
gitlab_rails['omniauth_providers'] = [
    {
      name: 'saml',
      args: {
               assertion_consumer_service_url: 'https://gitlab/users/auth/saml/callback',
               idp_cert_fingerprint: 'd5:bc:69:b2:f0:cb:0d:36:20:cd:c9:22:8f:2a:17:eb:53:58:c8:e6',
               idp_sso_target_url: 'https://idp/sso/login/gitlab',
               idp_slo_target_url: 'https://idp/sso/logout/gitlab',
               issuer: 'https://gitlab',
               name_identifier_format: 'urn:oasis:names:tc:SAML:2.0:nameid-format:persistent'
             },
      label: 'Login' # optional label for SAML login button, defaults to "Saml"
    }
  ]

@manu-silicon In fact, how do you obtain the idp_cert_fingerprint ?

I used the cert from the signing key on the IdP and then got the SHA1 of that. I used cat cert | openssl -sha1.
Manu

From: Tony Ngan<EMAIL_ADDRESS>Sent: Wednesday, May 10, 2017 8:08:37 AM
To: tngan/express-saml2
Cc: Manu; Mention
Subject: Re: [tngan/express-saml2] Guidance on certificate (#62)
@manu-siliconhttps://github.com/manu-silicon In fact, how do you obtain the idp_cert_fingerprint ?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/tngan/express-saml2/issues/62#issuecomment-300512697, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AMlVpBZChVqdogj7_tZLhy90Fn-PVpqoks5r4dL1gaJpZM4NUz2P.

@manu-silicon Nice, I will keep you posted in this thread so don't close this issue.

I also tried this to get the idp_cert_fingerprint:
openssl x509 -noout -fingerprint -sha1 -inform pem -in signing.crt
On the gitlab side, it is also not clear which certification you are supposed to use. I tried, the https one, the signing or the encryption crt and non seems to have an effect.

Yes, I am going to test the SAML approach using dockerize gitlab within this week, not sure if it restricts to https.

@manu-silicon Let me close this issue first, I will update the progress in the thread #12 within today.

@manu-silicon I have made some progress in it, hopefully it can be resolved soon. You can checkout here. #12


image/gif: EncodeAll fails to allow non looping animated gif

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (go version)?
go version go1.6.2

What operating system and processor architecture are you using (go env)?
darwin/amd64

What did you do?
In package image/gif, I'm trying to use EncodeAll() to create a non-looping animated gif with several frames, but it will always loop. In the GIF specifications 0 means forever, so -1 should the way to go to disable looping, unfortunately setting this value does not work in go.

What did you expect to see?
I should be able to set LoopCount = -1 and my resulting gif should not loop.
I want an animated gif, but not a looping animated gif.

What did you see instead?
My output gif is looping forever (same as LoopCount = 0).

The first problem lies in src/image/gif/write.go
   299		if g.LoopCount < 0 {
   300			g.LoopCount = 0
   301		}

This should be removed to allow a negative value.
The second problem is that Netscape Looping extension header is written in the file whenever there are several frames:
   147		if len(e.g.Image) > 1 {

I believe this should be changed to:
if len(e.g.Image) > 1 && e.g.LoopCount >= 0 {

Another way to fix the problem would be to change the meaning of LoopCount '0' value to actual "no loop", but this could break the expected behaviour for some people. As in other languages, setting -1 sounds ok as long as it's commented somewhere.

@nigeltao

@chriwup I believe what you want to do is set the loop count to 1 and not -1. IMO -1 doesn't make sense for a loop count to mean no loop. I have a small script that you can use to play around with the loop count values here https://play.golang.org/p/-yFYMIVPhZ and using 1 should cut the deal. Here is the code below inlined
package main

import (
	"flag"
	"fmt"
	"image/gif"
	"net/http"
	"os"
)

func exitIfError(err error) {
	if err == nil {
		return
	}

	fmt.Fprintf(os.Stderr, "%v\n", err)
	os.Exit(1)
}

func main() {
	gifURL := flag.String("url", "https://brobible.files.wordpress.com/2016/03/joker1.gif", "the URL of the GIF")
	loopCount := flag.Int("loop-count", 1, "the max loop count of the GIF")
	res, err := http.Get(*gifURL)
	exitIfError(err)
	gifImg, err := gif.DecodeAll(res.Body)
	_ = res.Body.Close()
	exitIfError(err)

	gifImg.LoopCount = *loopCount
	_ = gif.EncodeAll(os.Stdout, gifImg)
}

$ go run gif-loop-count.go --url https://j.gifs.com/Z6ozLJ.gif --loop-count 2 > lc2.gif # Make it loop twice
$ go run gif-loop-count.go --loop-count 1 > lc1.gif # Should loop only once.

Let me know what you think.

Actually I stand corrected, loop=1 sets it to loop once which is not what you want. Your original issue still stands, I've also corrected my script to include flag.Parse()

Yeah, it's unfortunate that, in the underlying "Netscape Looping Application Extension" format, 0 means loop forever and 1 means loop twice. To loop exactly once, we need to omit the extension entirely.
Given the back-compat constraints, I think that "negative means loop exactly once" is the right fix, in Go 1.8.

Punting to Go 1.9.
I started to do this but it seems important to harmonize both Read and Write, and it was not entirely clear to me how to do that.


[5.1] IRGen: Backport workaround for rdar://problem/53836960

Explanation: The Swift 5.1 version of LLVM miscompiles relative references to lazy_ptr symbols for symbols
defined in the same translation unit. Work around that by making GOT equivalents not appear as
GOT equivalents to the backend by withholding unnamed_addr.
Scope: Fixes a miscompile that leads to runtime crashes
Issue: rdar://problem/55082864
Testing: Swift CI
Risk: Low. We've used the same workaround for several other similar issues in the past.
Reviewed by: TBD

Replace this paragraph with a description of your changes and rationale. Provide links to external references/discussions if appropriate.

Resolves SR-NNNN.

@swift-ci Please test

@swift-ci Please nominate


fix: Fix the case, one service account has multiple roles

/cherry-pick release-1.0

/cherry-pick release-1.0


steam_input.rules: Use "uaccess" TAG

This will limit the users having access to these devices to the current user

Thank you, I've put the current set of udev rules into master and applied those changes to a beta branch. We will use those in the next steam package update.


flatpak: Fix launching flatpaks with DBusActivatable=true

Description of changes

Fixes #138956
At the moment, flatpak packages containing the line DBusActivatable=true fail to launch from both the app menu or via the gtk-launch command. (For example org.gnome.design.Palette)
journalctl output:
Mar 22 13:06:49 snowflakeos systemd[10420]<EMAIL_ADDRESS>Failed at step EXEC spawning flatpak: No such file or directory
░░ Subject: Process flatpak could not be executed
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░
░░ The process flatpak could not be executed and failed.
░░
░░ The error number returned by this process is ERRNO.

Things done
Added a patch to flatpak that defaults to /run/current-system/sw/bin/flatpak if the FLATPAK_BINARY variable is not found.

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
23.05 Release Notes (or backporting 22.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

I see no issue with https://github.com/NixOS/nixpkgs/pull/221408, only thing to note is that it is not a solution to https://github.com/NixOS/nixpkgs/issues/138956. I changed the patch so only change .service files that way it doesn't overlap with https://github.com/NixOS/nixpkgs/pull/221408 (although it would probably be best to combine the patch once one is merged).

How it is not solution to https://github.com/NixOS/nixpkgs/issues/138956? It changes the same hardcoded path export_desktop_file as your patch does. Or do the user services that would be produced by this do not respect PATH from the user profile (which would almost certainly contain flatpak)?

Or do the user services that would be produced by this do not respect PATH from the session environment (which would almost certainly contain flatpak)?

Correct, when the DBusActivatable=true is set, the PATH variable does not respect the sessions environment, rather than of systemd-path search-binaries-default.
From https://www.freedesktop.org/software/systemd/man/systemd.service.html#Command lines:

If the command is not a full (absolute) path, it will be resolved to a full path using a fixed search path determined at compilation time. Searched directories include /usr/local/bin/, /usr/bin/, /bin/ on systems using split /usr/bin/ and /bin/ directories, and their sbin/ counterparts on systems using split bin/ and sbin/. It is thus safe to use just the executable name in case of executables located in any of the "standard" directories, and an absolute path must be used in other cases. Using an absolute path is recommended to avoid ambiguity. Hint: this search path may be queried using systemd-path search-binaries-default.

In the case of NixOS, this only includes ${pkgs.systemd}/bin/, so when launched, these specific flatpaks fail to find the flatpak binary.

Ugh, I guess it is not really possible to have flatpak installed through Nix outside of NixOS.

My thought process was better to have it work on NixOS than to not work on either

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/some-flatpaks-fail-to-launch-from-launcher-icon/20818/6

flatpak.tests.installedTests failure is not new


Upgrade tt-metal(ium) level tests such as pgm dispatch benchmarks to use new benchmark data infra

What is the context?
There is some existing benchmarking infrastructure that can record performance results or benchmarks and upload them to a central Superset dashboard for later presentation and consumption.
The tools themselves were written by @skhorasganiTT and are housed in models/perf/benchmarking_utils.py. Infra team manages the infrastructure that transforms and submits the data to data teams' pipeline.
An example of use is here: falcon7b_common/demo/demo.py

Though it was written with models in mind, it should be easily applicable to runtime-level benchmarks.
What is there to do?
The current infrastructure is only supported in Python and was written with models in mind.
So, the following will likely need to be done by runtime team:

Get a feeling for what kind of insights Superset can offer
Decide what kind of benchmarks the team would like to see
Start making C++ versions of these benchmarking tools
Ensure data format compatibility with the current infrastructure (usually would mean Pydantic in Python, but need to come up with alternatives for C++ workflows w/ infra team)
Instrument current u-benchmarks to use these utilities

Do we use that? We should.
We can definitely edit the final schema however we choose to fit the requirements from data team.


Fix classList mutation methods

Skate SSR's implementation of classList does not make mutations to the className, this PR fixed the issues and adds tests.
If there is a linked issue, mention it here.

[x] Bug
[ ] Feature

Requirements

[x] Read the contribution guidelines.
[x] Wrote tests.
[x] Updated docs and upgrade instructions, if necessary.

Rationale
Fixes #1512
Implementation
I didn't try anything else. I didn't feel the need to try and micro-optimize this implementation for obvious reasons.
Open questions
What's going on with the deploy preview? It failed with failed during stage 'building site': Deploy directory 'site/public' does not exist. My changes should not have caused this.
Other
All these questions make contributing painful.

LGTM


Idea: Implement auto-reconect

I have some long-running rabbit consumers. Problem is that they after a some time loose connection to mysql (mysql end non-active connection) and it cause exception MySQL server has gone away
Here is a example of what I mean with auto-reconect:
https://github.com/facile-it/doctrine-mysql-come-back
As you can see that it is not that simple change as it was for example for Nette database.
Or if you know something much better way, please share it with me.
I know about this:
if ($this->entityManager->getConnection()->ping() !== TRUE) {
	$this->entityManager->getConnection()->close();
	$this->entityManager->getConnection()->connect();
}

but I think that something more systematic than copy+past this code in methods/repositories, will be better.
Thank you.

Hi. I get your point, but I don't know how to resolve it otherwise then https://github.com/facile-it/doctrine-mysql-come-back.

Closing it now. Feel free to open it and prepare PR. Thanks @novjan


Release location moved, crashing on recompile

If a reltool-generated release is moved (that is, if the entire directory structure is moved to a different location), running rebar from within that release (and using the release's Erlang version), it crashes.
The full error message is:
ERROR: compile failed while processing /home/gumm/www/nitrogen-git/myapp/lib/mimetypes: {'EXIT',{{badmatch,{error,enoent}},
         [{rebar_erlc_compiler,modify_erlcinfo,4,
                               [{file,"src/rebar_erlc_compiler.erl"},
                                {line,443}]},
          {rebar_erlc_compiler,'-update_erlcinfo_fun/2-fun-0-',4,
                               [{file,"src/rebar_erlc_compiler.erl"},
                                {line,399}]},
          {lists,foldl,3,[{file,"lists.erl"},{line,1261}]},
          {rebar_erlc_compiler,update_erlcinfo,3,
                               [{file,"src/rebar_erlc_compiler.erl"},
                                {line,418}]},
          {rebar_erlc_compiler,'-update_erlcinfo_fun/2-fun-0-',4,
                               [{file,"src/rebar_erlc_compiler.erl"},
                                {line,399}]},
          {lists,foldl,3,[{file,"lists.erl"},{line,1261}]},
          {rebar_erlc_compiler,init_erlcinfo,2,
                               [{file,"src/rebar_erlc_compiler.erl"},
                                {line,389}]},
          {rebar_erlc_compiler,doterl_compile,4,
                               [{file,"src/rebar_erlc_compiler.erl"},
                                {line,300}]}]}}

Upon further inspection is appears in trying to read the contents of a file that no longer exists:
In my case, the Source file generating the error is WARN: "/www/nitrogen-git/nitrogen/rel/nitrogen/lib/parsetools-2.0.12/include/leexinc.hrl"
This file was originally generated there, but the whole directory was moved to /www/nitrogen-git/myapp.  The naive approach is modifiying rebar_erlc_compiler:modify_erlcinfo/4 to not crash if the file does not exist seems to work, but it feels like a bandaid fix and not a real fix (and not totally knowing what exactly that module is doing, I was hoping someone with a better internal model of that code would know where a better fix would be applied).

@kejv, what do you think?

Given that each project has its own .rebar/erlcinfo, we should store the relative path instead.

I think we can't properly store relative paths because rebar just cds everywhere all the time, doesn't it? So if you're down in a dep, the path needs to be absolute to be properly readable from the project root later on, no?

Each project has its own .rebar/erlcinfo, so relative paths are fine for project-local files. External files like the parsetools header are a problem, and updating the entry with the new absolute path seems like the solution there.

I have another idea how to solve this. Let's see if it works out...

@choptastic can you try the wip-include_lib branch in my fork, and see if it helps?

@choptastic can you try the wip-include_lib branch in my fork, and see if it helps? If it works, I'd like to see a full log (-vv) to be sure.

I'll check of out within the hour
On Jun 3, 2015 2:51 PM, "Tuncer Ayaz"<EMAIL_ADDRESS>wrote:

@choptastic https://github.com/choptastic can you try the
wip-include_lib branch in my fork, and see if it helps? If it works, I'd
like to see a full log (-vv) to be sure.
—
Reply to this email directly or view it on GitHub
https://github.com/rebar/rebar/issues/503#issuecomment-108594769.

I'll add extra debug logs, please wait.

Logs modified.

@choptastic I need you to provide logs for three runs:

first run
second run, after moving stuff around
last run, rebuilding again without moving stuff around

@choptastic I need you to provide logs for three runs:

first run
second run, after moving stuff around
last run, rebuilding again without moving stuff around

It's probably best to put them in a gist.

Signing out for today.

Here's one big gist, captured with script and lightly annotated to differentiate each run.
As I understand your request, here's what I did, in short.

Build the nitrogen so that it's not moved around at all (the default nitrogen behavior is to move a new release to a different directory, so specifying the PROJECT and PREFIX locations as I did ensure they stay where they are)
Move the release to a new location on the filesystem: (mv rel/nitrogen ../myapp), cd into that dir, and run make (which just runs ./rebar get-deps compile)
Move the release back to its original location and run it again, demonstrating that it works in the original location.

https://gist.github.com/choptastic/e0fe1705ae3c1b815f2a

Also, I was a little wrong, since the app gets built and run from the released as a self-contained thing, the actual commands as executed from the Makefile looks more like this:
compile:
       @(export PATH=`pwd`/`echo erts-*/bin`:$$PATH; echo "Using Erlang in `which erl`"; $(REBAR) compile)

Which basically ends up running as:
export PATH=/www/nitrogen-git/nitrogen/erts-6.3/bin:$PATH; ./rebar -vv compile
This is to force the release to always use the release-copied version of Erlang.

I cannot see any include_lib message during ==> mimetypes (compile). Any idea?
Also, mimetypes/rebar.config seems to contain {erl_opts, [debug_info, fail_on_warning]}, but debug_info is enabled by default and fail_on_warning must be warnings_as_errors. So the correct option is {erl_opts, [warnings_as_errors]}.

I cannot see any include_lib message during ==> mimetypes (compile). Any idea?

Perhaps this can help.  Mimetypes doesn't use include_lib to include those headers, it uses a -file() directive, (I'm not sure what that is), and it's in a generated mimetypes_parse.erl file.
This is the mimetypes_scan.erl file that gets generated: https://gist.github.com/choptastic/d6fbec543e796c528a8e
And I'd guess it's crashing on this, which apparently gets hardcoded into the source code:
-file("/www/nitrogen-git/nitrogen/rel/nitrogen/lib/parsetools-2.0.12/include/leexinc.hrl", 14).
Also, thanks for the fixes for the rebar.config.

Okay, I see. You wrote

In my case, the Source file generating the error is WARN: "/www/nitrogen-git/nitrogen/rel/nitrogen/lib/parsetools-2.0.12/include/leexinc.hrl"

and from that it wasn't evident this is a -file() attribute. So, it's actually leex embedding attributes with absolute paths. From what I can tell, the same problem exists with yecc.
The logical solution seems to regenerate mimetypes_parse.erl if the absolute path becomes invalid.
We could try to detect a leex-generated file and manipulate the paths, but that's doesn't seem like rebar's responsibility. The same error would happen if you run erlc mimetypes_parse.xrl.

cc @rvirding @ferd

Nevertheless, can you please use the scenario to test the changes in my branch by adding an include_lib("kernel/include/file.hrl") to one of the files and repeat the steps (with -vv logs). That's because the bug which exists (but doesn't affect you yet) is real, and the branch should fix it.

from that it wasn't evident this is a -file() attribute. So, it's actually leex embedding attributes with absolute paths. From what I can tell, the same problem exists with yecc.

Agreed. I honestly didn't know what was causing it. It only first appeared after fixing #350

We could try to detect a leex-generated file and manipulate the paths, but that's doesn't seem like rebar's responsibility. The same error would happen if you run erlc mimetypes_parse.xrl.

The thing that strikes me here is that it works just fine with rebar 2.5.0. I haven't dug into any of the details about what the differences are between 2.5.0 and HEAD that would cause it to break.

Nevertheless, can you please use the scenario to test the changes in my branch by adding an include_lib("kernel/include/file.hrl") to one of the files and repeat the steps (with -vv logs). That's because the bug which exists (but doesn't affect you yet) is real, and the branch should fix it.

Sure, I'll give it a check in a bit.

The thing that strikes me here is that it works just fine with rebar 2.5.0. I haven't dug into any of the details about what the differences are between 2.5.0 and HEAD that would cause it to break.

That's because of the new dependency handling code. We process -file() attributes as dependencies of that source file, and the problem is that the path is absolute.
We can experiment with modifying the leex:file() call in rebar_erlc_compiler with a custom {includefile, "path_to_/leexinc.hrl"}:

You could try a relative path, but a directory move can break that as well.
You can try but I doubt -file() can be made to work like -include_lib() (which resolves the path via code:lib_dir()).
rebar could cache a copy of leex's and yecc's prologue header locally and always use a relative path via {includefile, ".rebar/leexinc.hrl"}

After doing the tests for the include_lib bug, manually cache leexinc.hrl somewhere local in mimetypes/ and modify the leex:file call in rebar to reference with a relative path. If that works, it's something we could automate in rebar, but I'm not sure we should.

Sorry missed the initial discussion. Leex adds these so that compiler
errors give the right file name, the use file name, and line numbers. Yecc
does the same. I don't really think it is that much of a need to do this
but they do.
Don't know how you would fix it though. What do you do with yecc generated
files?
Robert
On 4 June 2015 at 15:57, Tuncer Ayaz<EMAIL_ADDRESS>wrote:

The thing that strikes me here is that it works just fine with rebar
2.5.0. I haven't dug into any of the details about what the differences are
between 2.5.0 and HEAD that would cause it to break.
That's because of the new dependency handling code. We process -file()
attributes as dependencies of that source file, and the problem is that the
path is absolute.
We can experiment with modifying the leex:file() call in
rebar_erlc_compiler with a custom {includefile, "path_to_/leexinc.hrl"}:

You could try a relative path, but a directory move can break that
as well.
You can try but I doubt -file() can be made to work like
-include_lib() (which resolves the path via code:lib_dir()).
rebar could cache a copy of leex's and yecc's prologue header
locally and always use a relative path via {includefile,
".rebar/leexinc.hrl"}

After doing the tests for the include_lib bug, manually cache leexinc.hrl
somewhere local in mimetypes/ and modify the leex:file call in rebar to
reference with a relative path. If that works, it's something we could
automate in rebar, but I'm not sure we should.
—
Reply to this email directly or view it on GitHub
https://github.com/rebar/rebar/issues/503#issuecomment-109031481.

Sorry missed the initial discussion. Leex adds these so that compiler errors give the right file name, the use file name, and line numbers. Yecc does the same. I don't really think it is that much of a need to do this but they do.
Don't know how you would fix it though. What do you do with yecc generated files?

rebar takes -file() attributes into account when looking for a source file's dependencies to use for the needs-(re)compile check.
What happens here is that original location of the leexinc.hrl changes in a Nitrogren-based project after the initial generation of the parser from .xrl.
This crashes in rebar before the compile:file call because we fail to check the now missing leexinc.hrl. @choptastic any idea how this didn't crash before (old rebar) due to the missing file? What happens if you try to compile the file with erlc (after leexincl.hrl's is missing)?

@choptastic are you sure 2.5.0 doesn't crash? The same -file() handling code is present there as well.

Sorry missed the initial discussion. Leex adds these so that compiler errors give the right file name, the use file name, and line numbers. Yecc does the same. I don't really think it is that much of a need to do this but they do.
Don't know how you would fix it though. What do you do with yecc generated files?

rebar takes -file() attributes into account when looking for a source file's dependencies to use for the needs-(re)compile check.
What happens here is that original location of the leexinc.hrl changes in a Nitrogren-based project after the initial generation of the parser from .xrl.
This crashes in rebar before the compile:file call because we fail to file:open the now missing leexinc.hrl.

@tuncer, it does indeed compile fine with 2.5.0.  Here's the output of building it with rebar 2.5.0 after moving it away from its original location: https://gist.github.com/choptastic/52e04b6ec634e8e849b4

@choptastic are you sure mimetypes is compiled at least twice? I can only see one ==> mimetypes (compile).

To be clear, the old -file() attribute should fail, if you generated the file (compiled the project) before the dir-move.

In the script above there, it had already been compiled in the original location, hence why only 1 compile notification.
As far as compiling with erlc, it does compile even with the broken hard paths that don't exist:  Here is a relevant log of the console showing the file with the broken path, and it recompiling with erlc.
https://gist.github.com/choptastic/3c650b309d7b130d9296

erlc seems to ignore -file() attributes referencing non-existing files. We should do the same and log a DEBUG msg.

Please try the ignore-invalid-file-attr branch and post a -vv log.

Here's the run above with the include_lib patch (mimetype removed so it would succeed). There were no crashes.
I added -include_lib("kernel/file/include.hrl"). to site/src/index.erl
https://gist.github.com/choptastic/bda8deea55eb6ab3894a

Touched site/src/index.erl (using include_lib patch):
https://gist.github.com/choptastic/0572e046f42bee285bef

Using the ignore-invalid-file-attr branch, second run (after moving file locations):
https://gist.github.com/choptastic/d4cad2896bf2b3603eaa

The file gets cached in nitrogen/.rebar/erlcinfo and is expected to still exist after stuff is moved around. That's the failing file:open/2 call.

Please try the erlc-unstable-deps branch.

Please wait, there's other bugs.

I need to incorporate more fixes. As of yet, unfinished...

Please try my erlc-unstable-deps branch. It's full of debug logs, and the re-expanded path is not updated in the digraph, but it should fix the incorrect file:open/2 call. I'll probably have a look at the digraph update problem on Sunday.

The current iteration of erlc-unstable-deps has the right idea, but the implementation is flawed. Back to the drawing board.

@choptastic, can you please test my dep-open-crash branch?

Confirmed to be working on initial run. I will produce a -vv output for you in a moment.

And current master does not work, right?

Initial build which builds the release and compiles, then copies to new directory: http://paste.ubuntu.com/11631818/
Second build which tries to run from the new location: http://paste.ubuntu.com/11631938/
And confirmed to be broken in master.

I know you've been stressing about this, so I want you to know I appreciate you fixing this, even if this isn't quite the complete fix you want (unless it is, in which case, awesome).
The following is also true: The release is built with the current master version of rebar (built, but subsequent makes from other directories fail). Then the rebar binary is replaced with your patched version. Then run make again with your new binary (which succeeds). Then copying the old (that is, master) version of rebar back into the project. Then running make succeeds. So your rebar must be cleaning up the erlcinfo file so that the old rebar still works.
Either way, it's building with your patch, so kudos for your hard work.

It's not what I aimed to achieve, but a simple fix which makes sure that we treat the error case correctly.
There was a missing newline in the WARN message, which I just fixed in the branch, so please update your rebar.
Also, what's the log after re-running the test (with the new rebar)? If the 3rd and 4th logs are good as, this should suffice for now.

pull request: #509

Thanks for the bug report and testing.

Of course. Happy to help.
I'll get the report with the new change for you a bit later today, as I'm
currently away from the computer.
On Jun 7, 2015 3:27 PM, "Tuncer Ayaz"<EMAIL_ADDRESS>wrote:

Thanks for the bug report and testing.
—
Reply to this email directly or view it on GitHub
https://github.com/rebar/rebar/issues/503#issuecomment-109797469.

Hi, I have just got into hits. @tuncer Please let me think for a minute about it before merging the pull request.

After reading through this issue it is still puzzles me how it can happen. Because when the assumed root cause is that erlcinfo is copied to new release location which unchanged node/file names, then I don't know how modify_erlcinfo/4 can be called without Source being an existing file. This can only happen when digraph:vertex(G, Source) in update_erlcinfo/3 returns false which means that Source doesn't come from the original graph.
I think the problem is somehow connected to this part of the compile log:
WARN:  File missing. Remove from dep graph:
/www/nitrogen-git/nitrogen/rel/nitrogen/lib/parsetools-2.0.12/include/leexinc.hrl
WARN:  Failed to open file: /www/nitrogen-git/nitrogen/rel/nitrogen/lib/parsetools-2.0.12/include/leexinc.hrl
{error,enoent}WARN:  Failed to open file: /www/nitrogen-git/nitrogen/rel/nitrogen/lib/parsetools-2.0.12/include/leexinc.hrl
{error,enoent}WARN:  File missing. Remove from dep graph:

Here it seems as if the leexinc.hrl file was added (and removed) several times to (and from) the dep graph which is quite strange.

@david-kubecka, I agree that modify_erlcinfo() shouldn't have to deal with this at all. To be clear, the patch is for unblocking the 2.6.0 release, as I ran out of time for implementing a proper fix. That's why I filed #506. If you agree this is fine to merge, we can discuss the proper solution there, and get the already belated release out the door.

@tuncer I understand (previously was writing from my company github account, sorry). The patch seems ok. I will copy my investigation to #506 so that it doesn't get lost.

@ferd just merged #509.

This can be closed.


Extending IndicBART or IndicBERT

I want to basically pre-train models from scratch, including tokenizer, for languages included in IndicBART and IndicBERT and some more languages, so as to build a something like IndicBARTExt and IndicBERTExt.
While going through some related issues, I noticed that there some conventions about language codes. It is possible to use pre-existing language codes for new languages.
Is there some way to add new language codes, say, for IndicBART/IndicBERT without much change in Python code which is called from the pre-train shell script? Or will it require considerable changes?

Or, if I can continue pre-training of the IndicBART and IndicBERT models on some more Indian languages, preferably with additional language codes, that will be even better. I have a vague idea of how to do it, but not exactly in terms of the toolkit code. Any help or pointers will be helpful.

There is already an indicbert v2 which supports 24 Indic languages: https://huggingface.co/ai4bharat/IndicBERTv2-SS
AFAIK it was trained from scratch.
As for IndicBARTv2, there is one in the works and should be out soon.
Regarding what you want to do, you will need to figure out the following:

Extend the AlbertTokenizer with additional language tokens.
Corresponding to the IDs of the language indicator tokens, extend the embedding matrix of the IndicBART model checkpoint (the .pt or .bin file containing the weights).

This will involve no change to the codebase.

The URL you included is giving 404 error. Is there some other place where it may be described?
Extend the AlbertTokenizer with additional language tokens.

I should be able to do this.
Corresponding to the IDs of the language indicator tokens, extend the embedding matrix of the IndicBART model checkpoint (the .pt or .bin file containing the weights)

For this can I simply load the pre-trained model and call the train script or function again on more data or should I keep in mind some other points?
And do the language codes matter or one can simply reuse existing codes? In the paper I remember reading that languages are not distinguished while training to allow zero shot learning. But I guess the code will matter when trying to actually translate.
BTW, I also want to fine-tune on some basic NLP tasks like POS tagging and NER etc. Will the pre-training differ for bilingual and monolingual tasks. There are, I think two different scripts for monolingual and translation tasks.

The URL is working now. I have data for three languages, two of them are not in the list of 24 and for one I may have some more data.
For IndicBERT, I have posted an issue on their repository.
For IndicBART, I am not clear about how to use the language codes to continue pre-training or while fine-tuning.
Should the pre-training be different for multilingual parallel corpora and multilingual monolingual corpora? Or will only fine-tuning be different? In either case, how to proceed properly? I don't have experience working with BERT before.

The URL you included is giving 404 error. Is there some other place where it may be described?
Extend the AlbertTokenizer with additional language tokens.

I should be able to do this.
Corresponding to the IDs of the language indicator tokens, extend the embedding matrix of the IndicBART model checkpoint (the .pt or .bin file containing the weights)

For this can I simply load the pre-trained model and call the train script or function again on more data or should I keep in mind some other points?

You will have to look into how to resize the embedding layer. You will have to hack YANMTT for this. Take a look here for hints: https://discuss.huggingface.co/t/adding-new-tokens-while-preserving-tokenization-of-adjacent-tokens/12604/3

And do the language codes matter or one can simply reuse existing codes? In the paper I remember reading that languages are not distinguished while training to allow zero shot learning. But I guess the code will matter when trying to actually translate.

You can reuse but its a hacky solution.

BTW, I also want to fine-tune on some basic NLP tasks like POS tagging and NER etc. Will the pre-training differ for bilingual and monolingual tasks. There are, I think two different scripts for monolingual and translation tasks.

YANMTT is not designed for basic NLP tasks. Its designed for NLG tasks. However you can treat the NLP task as a NLG task and see what happens. One script is for pre-training however, the fine-tuning script can also be indirectly used for pre-training. Im planning to retire the pre-training script since the latter one can already do everything.

The URL is working now. I have data for three languages, two of them are not in the list of 24 and for one I may have some more data.
For IndicBERT, I have posted an issue on their repository.
For IndicBART, I am not clear about how to use the language codes to continue pre-training or while fine-tuning.

For fine-tuning you can take a look here: https://github.com/AI4Bharat/indic-bart
For continued pre-training, you can reuse the commands in the aforementioned repo except that your source and target corpora are the same monolingual corpora files.

Should the pre-training be different for multilingual parallel corpora and multilingual monolingual corpora? Or will only fine-tuning be different? In either case, how to proceed properly? I don't have experience working with BERT before.

Rather than jumping into continued fine-tuning I recommend that you first get used to pretraining models from scratch with YANMTT. Once you familiarize yourself with this, things will get easier. Look into the examples folder for help.

Also I recommend going through the issues section. Most of your questions can be answered from there:
https://github.com/prajdabre/yanmtt/issues/37
https://github.com/prajdabre/yanmtt/issues/36
https://github.com/prajdabre/yanmtt/issues/34
https://github.com/prajdabre/yanmtt/issues/33
https://github.com/prajdabre/yanmtt/issues/54
https://github.com/prajdabre/yanmtt/issues/31
https://github.com/prajdabre/yanmtt/issues/17
https://github.com/prajdabre/yanmtt/issues/5


Pass camera orientation to OpenALPR?

This might improve the recognition rate, but I'm not sure if that's a problem yet.
See http://doc.openalpr.com/accuracy_improvements.html#calibration (but note that the Cloud API does detect the plate in the "before" image)
Not sure if the orientation data (yaw/pitch/roll, or alpha/beta/gamma) is part of EXIF (https://stackoverflow.com/questions/41674012/metadata-extractor-orientation-gyroscope),
but if the picture is taken in-app, we could potentially grab them using deviceorientation events:

https://developer.mozilla.org/en-US/docs/Web/API/Detecting_device_orientation
https://developers.google.com/web/fundamentals/native-hardware/device-orientation/

On a related note, it turns out the OpenALPR Cloud API does not handle EXIF orientation, so I fixed it server-side in 38b28bb67d0249320bdf5967ebc22ebd6e5fcb41

Not sure if the orientation data (yaw/pitch/roll, or alpha/beta/gamma) is part of EXIF

It doesn't seem to be, at least for images taken by my phone.

if the picture is taken in-app, we could potentially grab them using deviceorientation events.

This may be true, but taking the picture in-app doesn't preserve the EXIF data for location (and possibly time), so it's less useful than uploading a previously taken picture. See here for more on the EXIF:

https://stackoverflow.com/questions/14967559/losing-gps-metadata-while-uploading-picture-with-mobile-safari
https://stackoverflow.com/questions/36221538/exif-data-being-truncated-when-taking-a-photo-via-phone-to-upload/36263805#36263805

Closing this for these reasons, especially since the OpenALPR Cloud API seems pretty robust, as mentioned above:

See http://doc.openalpr.com/accuracy_improvements.html#calibration (but note that the Cloud API does detect the plate in the "before" image)


CWD on tmux split

What system are you running Yazi on?
macOS
What terminal are you running Yazi in?
Ghostty
yazi --debug output
Yazi
    Version: 0.4.1 (Homebrew 2024-12-10)
    Debug  : false
    Triple : aarch64-apple-darwin (macos-aarch64)
    Rustc  : 1.83.0 (90b35a62 2024-11-26)

Ya
    Version: 0.4.1 (Homebrew 2024-12-10)

Emulator
    Brand.from_env      : Some(Ghostty)
    Emulator.detect     : Emulator { kind: Left(Ghostty), light: false, cell_size: Some((9, 19)) }
    Emulator.detect_full: Ok(Emulator { kind: Left(Ghostty), light: false, cell_size: Some((9, 19)) })

Adapter
    Adapter.matches: Kgp

Desktop
    XDG_SESSION_TYPE           : None
    WAYLAND_DISPLAY            : None
    DISPLAY                    : Some("/private/tmp/com.apple.launchd.F1LGnP0z60/org.xquartz:0")
    SWAYSOCK                   : None
    HYPRLAND_INSTANCE_SIGNATURE: None
    WAYFIRE_SOCKET             : None

SSH
    shared.in_ssh_connection: false

WSL
    WSL: false

NVIM
    NVIM          : false
    Neovim version: 0.10.2

Variables
    SHELL           : Some("/bin/zsh")
    EDITOR          : Some("nvim")
    VISUAL          : Some("nvim")
    YAZI_FILE_ONE   : None
    YAZI_CONFIG_HOME: None
    YAZI_ZOXIDE_OPTS: None
    FZF_DEFAULT_OPTS: Some("--height 10")

Text Opener
    default     : Some(Opener { run: "${EDITOR:-vi} \"$@\"", block: true, orphan: false, desc: "$EDITOR", for_: None, spread: true })
    block-create: Some(Opener { run: "${EDITOR:-vi} \"$@\"", block: true, orphan: false, desc: "$EDITOR", for_: None, spread: true })
    block-rename: Some(Opener { run: "${EDITOR:-vi} \"$@\"", block: true, orphan: false, desc: "$EDITOR", for_: None, spread: true })

Multiplexers
    TMUX               : 2
    tmux version       : tmux 3.5a
    tmux build flags   : enable-sixel=Unknown
    ZELLIJ_SESSION_NAME: None
    Zellij version     : No such file or directory (os error 2)

Dependencies
    file          : 5.41
    ueberzugpp    : No such file or directory (os error 2)
    ffmpeg/ffprobe: 7.1 / 7.1
    pdftoppm      : 24.12.0
    magick        : 7.1.1-41
    fzf           : 0.56.3
    fd/fdfind     : 10.2.0 / No such file or directory (os error 2)
    rg            : 14.1.1
    chafa         : 1.14.5
    zoxide        : 0.9.6
    7z/7zz        : 17.04 / No such file or directory (os error 2)
    jq            : 1.7.1

Clipboard
    wl-copy/paste: No such file or directory (os error 2) / No such file or directory (os error 2)
    xclip        : No such file or directory (os error 2)
    xsel         : No such file or directory (os error 2)

Describe the bug
Part of my workflow involves using yazi to navigate to a subdirectory  of a project and then opening a tmux split, this way I get a terminal on the bottom and I can open a file with nvim on the top, before upgrading, the tmux split would open the directory I was on in yazi, now it's opening on the directory where I started yazi from (the root dir of the project)
I've tested other file browsers, like lf and ranger and they behave as I expect, the tmux splits are open on the folder I've navigated to.
Note that I'm not using any plugin or shell integration, I also expect that when I exit yazi, I get back to the directory I started from, like before and like with other similar file browsers.
Minimal reproducer
start yazi in tmux, navigate to a sub-folder, open a tmux split.
The expectation is that the split opens in the directory I navigated to (old behaviour before upgrading to the latest version), instead the split opens on the directory yazi was started from.
Anything else?
No response
Checklist

[X] I tried the latest nightly build, and the issue is still reproducible
[X] I updated the debug information (yazi --debug) input box to the nightly that I tried
[X] I can reproduce it after disabling all custom configs/plugins (mv ~/.config/yazi ~/.config/yazi-backup)

Please try https://github.com/sxyazi/yazi/pull/2043

Confirmed that https://github.com/sxyazi/yazi/pull/2043 fixes the issue, thanks for getting to it so fast.


test failures in master

There are several test failures in master

com.bluelinelabs.logansquare.processor.MultipleFieldNamesModelTest > generatedSource FAILED
    java.lang.AssertionError at MultipleFieldNamesModelTest.java:18

com.bluelinelabs.logansquare.processor.SimpleModelTest > generatedSource FAILED
    java.lang.AssertionError at SimpleModelTest.java:18

com.bluelinelabs.logansquare.processor.SimpleCollectionModelTest > generatedSource FAILED
    java.lang.AssertionError at SimpleCollectionModelTest.java:18

com.bluelinelabs.logansquare.processor.PrivateFieldModelTest > generatedSource FAILED
    java.lang.AssertionError at PrivateFieldModelTest.java:18

They're all passing for me. Any idea what the errors are exactly? Gradle should produce a .html file with actual stack traces.

LoganSquare expects you to use Java 7. I got these errors too when using JDK 8.

I'll close this for now, since the tests do pass using Java 7. I'll circle back to Java 8 compatibility once I get a little more time. If you can post the html that gradle generates with all the errors, that would be really helpful.


Updating link from old tox issue tracker to new one.

Codecov Report

Merging #27 into master will not change coverage.
The diff coverage is n/a.

@@          Coverage Diff          @@
##           master    #27   +/-   ##
=====================================
  Coverage     100%   100%
=====================================
  Files          11     11
  Lines         511    511
  Branches       75     75
=====================================
  Hits          511    511

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 22204b2...9b96aa8. Read the comment docs.


ADD a continuous image puller

For a continuous image puller, I believe we can follow the lead of what jupyterhub does.
https://github.com/jupyterhub/zero-to-jupyterhub-k8s/tree/main/jupyterhub/templates/image-puller
https://github.com/jupyterhub/zero-to-jupyterhub-k8s/tree/main/images/image-awaiter


Swap wstx to velar KO

Describe the bug
Hi, I've tried many times to swap some wstx token (tesnet) to velar using https://app.velar.co/swap, but all my transactions failed. Could you please explain me why ??
Transaction ID 0x924dbde872d5afd4042d74536724facb92838f118424e2b75b7cac0a2f33d10b
Address ST1MC6G0H5EGCP43BBVJTAACBKPYVDNFNNHHEXH14
Block#  0x6ce67da90f9b0ad9777e71a1da9154ff49f10740b9b3a3d60bd2ea35d4c09179
Time stamp
To Reproduce
Steps to reproduce the behavior:

Go to 'https://app.velar.co/swap'
Click on 'Swap'
Enter an amount of 'wstx'
Confirm transaction to swap token.

Expected behavior
I'm expecting that the transaction is successful after some minutes, but everytime it fails.
Desktop (please complete the following information):

Browser chrome

This doesn't seem like an issue in the Expolorer, maybe you can ask help in Discord https://discord.gg/pw4kGYuK


[staging] perl: 5.38.2 -> 5.40.0

Hydra Jobset: https://hydra.nixos.org/jobset/nixpkgs/perl-updates

Update perl to 5.40.0
Update perl-cross to 1.6
Remove perl536
Update cross and no-sys-dirs patches
Fix for #295608 from @tobim
⚠ Set enableParallelBuilding=false as unicore/mktables step apparently consumes a lot of resources with large parallel builds, causing the build to fail sometimes

Ref #292588 (Cc @Atemu)
TODO: reproduce reliably and find better workaround

⚠ Use perl538 in stdenv/linux/default.nix since perl540 fails to build in stage1
Updates to other packages:

macvim: 178 -> 179, perl536 -> perl540
unit: add nixos tests for perl, clean up withPerl arguments
perlPackages.AuthenModAuthPubTkt: disable unstable test
perlPackages.CGICompile: disable unstable test
perlPackages.ApacheDB: mark as broken
perlPackages.MouseXGetOpt: remove failing tests to fix build
perlPackages.EncodeIMAPUTF7: patch to fix build with 540
perlPackages.XSParseKeyword: 0.38 -> 0.44
perlPackages.ObjectPad: 0.804 -> 0.809
perlPackages.TextLayout: 0.031 -> 0.037
perlPackages.FileLoadLines: 1.021 -> 1.046
perlPackages.MooseXGetopt: 0.75 -> 0.76
perlPackages.DevelSize: 0.83 -> 0.84
perlPackages.CodeTidyAll: 0.83 -> 0.84
perlPackages.AppMusicChordPro: 6.030 -> 6.050.7
perlPackages.po4a: 0.71 -> 0.73
perlPackages.GetoptLongDescriptive: 0.111 -> 0.114
perlPackages.BKeywords: 1.26 -> 1.27

Add a :+1: reaction to pull requests you find important.

I opened #338838 targetting this PR to fix #295608.

Barring any objections, I'd like to merge this in the next day or so.

⚠ Set enableParallelBuilding=false as unicore/mktables step apparently consumes a lot of resources with large parallel builds, causing the build to fail sometimes

I don't like the idea of disabling parallelism because people spec their hardware disproportionally or configure their nix-daemon wrong. That should be reserved for cases where the build system has an actual bug due to parallelism that is independent of the build host.
IIRC Make has a feature to limit the maximum load it can put on the system. Couldn't this be used instead?
There are stalled plans to make this available as a first-class Nix sandbox setting similar to NIX_BUILD_CORES but there's no reason we couldn't just set this to some reasonable value in the meantime.

@Atemu I've tested building by setting a max value for $NIX_BUILD_CORES to 8, this appears to work, but needs to be tested a bit more with cross and static build. Does this seem like a reasonable implementation?
diff --git a/pkgs/development/interpreters/perl/interpreter.nix b/pkgs/development/interpreters/perl/interpreter.nix
index b95adaf8287d..20364a145925 100644
--- a/pkgs/development/interpreters/perl/interpreter.nix
+++ b/pkgs/development/interpreters/perl/interpreter.nix
@@ -148,7 +148,7 @@ stdenv.mkDerivation (rec {

   dontAddPrefix = !crossCompiling;

-  enableParallelBuilding = false;
+  enableParallelBuilding = true;

   # perl includes the build date, the uname of the build system and the
   # username of the build user in some files.
@@ -210,6 +210,18 @@ stdenv.mkDerivation (rec {
       perlOnTargetForTarget = if lib.hasAttr perlAttr pkgsTargetTarget then (override pkgsTargetTarget.${perlAttr}) else { };
     };

+  buildPhase = ''
+    runHook preBuild
+
+    if (( $NIX_BUILD_CORES > 8)); then
+      NIX_BUILD_CORES=8
+    fi
+
+    make -j$NIX_BUILD_CORES
+
+    runHook postCheck
+  '';
+
   doCheck = false; # some tests fail, expensive

   # TODO: it seems like absolute paths to some coreutils is required.

What was the hurry with merging this while there was discussion ongoing? We're still in the middle of staging-next-24.05, so this could absolutely have waited a couple of days to let the discussion play out.

While I'd prefer the issue of parallel builds to be discussed, it's a minor issue all things considered; we can live with single-threaded perl builds for a staging cycle or two.

I agree that it's not a big deal.  Just seemed odd to me to merge it while discussion was still happening when the next staging cycle won't start for days anyway.

In the commit message, I wrote this:

After discussing on Matrix in the #staging room, I'm merging this as-is without waiting for changes to the enableParallelBuilding = false; line.
There are several in-the-wings PRs (notably one from @emilazy) that attempt to bring load-limit, which is a more sensible accounting of the work that a derivation is doing, into Nix itself. If and when that lands, we'll be able to just set enableParallelBuilding = true; again, without any specific casing around the NIX_BUILD_CORES stdenv variable.

I'm realizing that this message is not surfaced in the PR or that discoverable other than by hovering over the hash, so I'm committing to writing something on a PR if I do a merge commit message like that again.

More generally, I'd like be part of the solution to a problem many, many, many contributors to Nixpkgs say they have: they don't know where the "finish line" is for merging their PR, and which feedback is blocking and which is just a general discussion or dissatisfaction about the state of the world.
And so as a result as their limited capacity for discussion, or to keep up with the tax of keeping a PR fresh, many excellent contributions have died on the vine, despite no one in the PR process actually deciding to block the PR for a specific reason.
That's not a worry here. Perl 5.40 is going to happen one way or another. So making this be the place to make a stand isn't the wisest decision I've ever made...
What I'd like to do is help us, the collective who keeps Nixpkgs humming, become better about crisply stating objections by making an ask to contributors to do that and, failing to hear objections that state that they block the PR, merging it.
I'm very open to feedback about how I do this, about the length of time for a debate or waiting for contributors to have their say. Definitely going to get this wrong some of the time.
I'm much less open to practices that end up contributing to a culture of endless debate without clear messages like "I oppose merging this PR because it does XYZ, and XYZ shouldn't be done." The request changes button is right there, it's useful to ask people to click it when needed, otherwise, let's merge and iterate.
Again, definitely might have gotten that wrong in this case. Based on @atemu's message above, it sounds like they didn't consider their feedback blocking, so I do feel like it's a fair move to merge, but I am open to feedback about how I ought to behave in the future.

Btw, we don't need to wait for the load limit PR to land. That PR is only about adding generic support that can be used tree-wide in a manner that fits all users.
We can just set a load limit to a constant relative to the available CPU resources here right now and it'd be merely slightly suboptimal at worst in some edge-cases. A hellofalot better than disabling paralellism outright.

Load limits were dropped across the board in https://github.com/NixOS/nixpkgs/pull/192447 because they make things worse for Hydra. I think patchwork solutions here are probably not a great fix, since it’s hard to pick a value that works well for a wide variety of systems, and since load limit looks at the total system load, not the load actually being used by a build.
FWIW I support just enabling the parallel build again here in a separate PR since it doesn’t cause issues for Hydra and there are local workarounds.

More generally, I'd like be part of the solution to a problem many, many, many contributors to Nixpkgs say they have: they don't know where the "finish line" is for merging their PR, and which feedback is blocking and which is just a general discussion or dissatisfaction about the state of the world.
And so as a result as their limited capacity for discussion, or to keep up with the tax of keeping a PR fresh, many excellent contributions have died on the vine, despite no one in the PR process actually deciding to block the PR for a specific reason.
That's not a worry here. Perl 5.40 is going to happen one way or another. So making this be the place to make a stand isn't the wisest decision I've ever made...

Yeah, I think that's exactly it.  This is a great attitude to have — it just didn't really apply here.  All I'd have liked to see being different was waiting a few days, since it's a staging PR anyway so all that matters is that it catches the next cycle.

@emilazy my point is that disabling parallelism entirely is even worse for hydra.
Making this tunable is only required for enabling load limits globally. It is not a requirement for applying it in targeted cases like this one.

perl536 attribute removal has a small perlInterpreters eval fallout. Attempting to fix it as:

https://github.com/NixOS/nixpkgs/pull/341985


Update-Conflict related to an API error and Lambda not supporting node8.10 any longer

Describe the bug
Updating the Amplify-CLI version to 4.18.1 brought my amplify-based web-app into a corrupt state: The CloudFormation stack is stuck in the state UPDATE_ROLLBACK_FAILED that I need to resolve manually. Besides reporting the issue I would need some guidance on resolving the issue manually without data loss.
I updated my amplify cli to the new version. Upon first invocation of amplify status I answered the prompt to update the Nodejs runtime from node8.10 to node10.x with Yes.
I then invoked amplify push to update the app's lambdas and cognito (that has the inline lambda UserPoolClientLambda).
During deployment, there then was an unrelated UPDATE_FAILED error in my GraphQL API component, for which there is a closed amplify-cli issue #2626 here, the error message being Parameters: [CreateAPIKey] do not exist in the template.
CloudFormation then attempted a rollback, which failed, because the old runtime of the lambdas node8.10 is not supported by AWS Lambda anymore.
As per Amplify docs the resulting UPDATE_ROLLBACK_FAILED state needs to be resolved manually. According to what I have read, the only way to get out of this state is to either complete the rollback (which in this case is not possible, because the template I want to roll back to cannot be changed and the settings in that template are not supported anymore) or to delete the stack. (Is this understanding correct?)
This happened to me in my test env, so I do not have a problem with deleting that stack. The prod env is still untouched and runs the node8.10 runtime. The problem is that the User accounts of our customers that cognito holds would be lost when we delete the prod stack, so we need a reliable way of updating the prod env, but no way to test because we do not have a working replica of the prod stack anymore.
Amplify CLI Version
4.18.1

@peterjungx did your API have any change to be pushed before the migration script got triggered?
cc @UnleashedMind

No. I did not make any changes to the schema and amplify status showed No Change in the Operation column in the Api categories.

Try to use the steps pointed out in this issue #3792 to first bring the stack back to a COMPLETE status, we can then work to upgrade the node version.

@UnleashedMind Thank you for the response! You pointed me in the right direction and the problem is solved.
The steps here https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/using-cfn-updating-stacks-continueupdaterollback.html#nested-stacks did bring me on the right track. I gained a better understanding of what is going on in continue update rollback. This needed three iterations to complete, but I am there.
Afterwards I could solve the update issue with the extra space in the schema as per https://github.com/aws-amplify/amplify-cli/issues/2626 to force template recalc on that part.
Thanks!


Upgrade test dependencies & fix Falcon warning

This pull request upgrades the dependencies in testing. Also, it resolves a deprecation warning in Falcon 3.x.
Category (place an x in each of the [ ])

[ ] slack_bolt.App and/or its core components
[ ] slack_bolt.async_app.AsyncApp and/or its core components
[x] Adapters in slack_bolt.adapter
[ ] Document pages under /docs
[x] Others

Requirements (place an x in each [ ])
Please read the Contributing guidelines and Code of Conduct before creating this issue or pull request. By submitting, you are agreeing to those rules.

[x] I've read and understood the Contributing Guidelines and have done my best effort to follow them.
[x] I've read and agree to the Code of Conduct.
[x] I've run ./scripts/install_all_and_run_tests.sh after making the changes.

👋 Let me know if you have any comments on this

Thanks for the review!


Generalized content processing with external libraries

A common task for processing tilesets is: "Apply a certain operation to every tile content". Some distinctions here are whether this operation changes the name (content.uri) of the tile content or not, whether it can "remove/delete" the content or not, or whether it has to apply special handling depending on the type of the content data, or whether it requires additional information (configuration options) at runtime.
There already are some structures that support all these considerations. But a frequent pattern is that all these specific questions do not matter (or are inherently answered).
One example is when optimizeGlb is called on all GLB contents of a tileset: It does not change URIs. It always operates on GLB data. It just runs gltf-pipeline on all contents, and does whatever gltf-pipeline does.
Other operations could fit into this simple pattern as well. For example, integrating meshoptimizer to support meshopt compression. Or operations that modify the textures inside a glTF. Or ... running gltf-transform, which is basically a combination of all the above: It allows compression, texture manipulation, or other operations on GLB data. But it always follows the pattern "GLB in - GLB out".
This PR adds some convenience classes that allow an easier integration of libraries like that (and uses gltf-pipeline and gltf-transform for now).
A draft usage example is shown here: It applies a set of operations to each (GLB) tile content:

Use gltf-transform to...

prune unused elements
deduplicate elements
(apply draco compression - for tests, commented out here)
apply a texture resizing/compression (delegating to the sharp library)

Then use gltf-pipeline to apply draco compression

import {
  prune,
  dedup,
  draco,
  textureCompress,
} from "@gltf-transform/functions";
import sharp from "sharp";

import { TileContentProcessing } from "./src/tilesetProcessing/TileContentProcessing";
import { TileContentProcessors } from "./src/tilesetProcessing/TileContentProcessors";
import { TileContentProcessorsGltfTransform } from "./src/tilesetProcessing/TileContentProcessorsGltfTransform";
import { TileContentProcessorsGltfPipeline } from "./src/tilesetProcessing/TileContentProcessorsGltfPipeline";

async function runTest() {
  const tilesetSourceName = "./input/TestTileset";
  const tilesetTargetName = "./output/TestTileset";
  const overwrite = true;

  // Some `Transform` instances that describe operations
  // that are executed with `gltf-transform`
  const transformDraco = draco({
    method: "edgebreaker",
    encodeSpeed: 5,
    decodeSpeed: 5,
    quantizeGeneric: 10,
  });
  const transformTextures = textureCompress({
    encoder: sharp,
    targetFormat: "jpeg",
    quality: 70,
    resize: [512, 512],
  });

  // Create a `TileContentProcessor` that applies
  // some transforms with `gltf-transform`
  const tileContentProcessorGltfTransform =
    TileContentProcessorsGltfTransform.create(
      dedup(),
      prune(),
      //transformDraco,
      transformTextures
    );

  // Create a `TileContentProcessor` that applies
  // `gltf-pipeline` with the given options
  const gltfPipleineOptions = {
    dracoOptions: {
      compressionLevel: 10,
    },
  };
  const tileContentProcessorGltfPipeline =
    TileContentProcessorsGltfPipeline.create(gltfPipleineOptions);

  // Combine several `TileContentProcessor` instances
  // into one
  const tileContentProcessor = TileContentProcessors.concat(
    tileContentProcessorGltfTransform,
    tileContentProcessorGltfPipeline
  );

  // Process the tileset source, and write it to the tileset target,
  // applying each `TileContentProcessor` to all tile contents
  await TileContentProcessing.process(
    tilesetSourceName,
    tilesetTargetName,
    overwrite,
    tileContentProcessor
  );
}

runTest();

Many of the structures may still change, but this may already be close to the pseudocode of
processTileContent(input, {
  doThis(),
  doThat(withOptions),
  thenDoAnotherThing()
});

with some flexibility for what is actually done with the tile content data.
There's still the caveat that this is currently only possible in code: There is not yet a sensible pipeline.json representation of all that. Parts of this could easily be mapped to specific pipeline JSON representations. The question then is: How specific can this JSON representation be? (It should, for example, not "know" that a texture resize is done with gltf-transform that delegates to the sharp library...)

The functionality that was initially drafted here is now part of https://github.com/CesiumGS/3d-tiles-tools/pull/41 . An example of how to use glTF-Transform, gltfpack, and gltf-pipeline (similar to the snippet shown above) has been added as a TileContentProcessingDemo


Support light/dark logo on README

What happened 👀
Quick update for the light/dark logo visibility.
Insight 📝
Minor update
Proof Of Work 📹
Before:
https://user-images.githubusercontent.com/13435717/173321532-2efe3e86-28ff-4559-8bbe-cdb3e924c5bb.mov
After:
https://user-images.githubusercontent.com/13435717/173321405-40354c78-1f3d-4ed5-98bc-77b5b84ce9f7.mov

@sleepylee Why is there a Jacoco report on this PR? 😆

also support the footer logo


Version for Kubernetes-1.10

Hi,
Can you please release a version for Kubernetes-1.10 release? Also, is kubebuilder supposed to superseded this tool or kubebuilder is for CRDs and this one is for EAS-es?
Thanks.

hey @tamalsaha, are you still hitting this issue? release-1.10 we can provide a release-1.10 if that helps.

Also, is kubebuilder supposed to superseded this tool or kubebuilder is for CRDs and this one is for EAS-es?

long-termly, i suppose it's a yes. while for now, CRDs have some pending underlying issues so.. the apiserver builder will move on

I would love this. I'm using this library and would love to upgrade to kube apimachinery 1.10 to pick up a bunch of advancements in CRDs, but I can't go all the way to 1.12 (which is what has happened on master).

We are not planning to use this. Thanks for checking.

/assign