Signing multiple references and non-XML content in detached mode

Hi
I am trying to port some C code to python that is signing an XML referencing another document, a so called detached signature.
I was told that it is possible to specify "detached" as signature method when creating XMLSigner.
But then I expected that I also had to supply an uri-resolver method to the sign method just like with the verify method to supply the data for the uri specified. Something like the following:
def my_uri_resolver(self, uri):
    if uri == 'uri1':
        return 'data for uri1'
    if uri == 'uri2':
        return 'data for uri2'
...
signer = XMLSigner()
signed = signer.sign(
    unsigned_xml,
    key=my_key,
    passphrase=my_passphrase,
    cert=my_cert,
    reference_uri=['uri1', 'uri2'],
    uri_resolver=my_uri_resolver)
...

Am I missing something here?
Regards Asbjørn Oskal

Quoting the README file:

signed_root = XMLSigner(method=signxml.methods.detached).sign(root, key=key, cert=cert)

For detached signatures, the code above will use the Id or ID attribute of root to generate a relative URI (<Reference URI="#value"). You can also override the value of URI by passing a reference_uri argument to sign(). To verify a detached signature that refers to an external entity, pass a callable resolver in XMLVerifier().verify(data, uri_resolver=...).
You pass the data to be referenced by the signature directly as the first argument.

Hi again and thanks for the response.
I am sorry to bug you but the scenario I want to solve is this:
I have two files. One envelope-xml and one payload (It can be any kind of file really. For example a gif-file)
I sign the envelope-xml with a reference to the gif-file so that in the envelope xml there will be two signature elements. One for the envelope itself  and one for the external file
The DigestValue for "cid:ref1" will be the hash of the external gif-file.
The envelope-xml should then end up with SignedInfo content like this:
...
<SignedInfo>
	<CanonicalizationMethod Algorithm="http://www.w3.org/TR/2001/REC-xml-c14n-20010315"/>
	<SignatureMethod Algorithm="http://www.w3.org/2000/09/xmldsig#rsa-sha1"/>
	<Reference URI="">
		<Transforms>
			<Transform Algorithm="http://www.w3.org/2000/09/xmldsig#enveloped-signature"/>
			<Transform Algorithm="http://www.w3.org/TR/1999/REC-xpath-19991116">
				<XPath xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/">not(ancestor-or-self::node()[@SOAP-ENV:actor="urn:oasis:names:tc:ebxml-msg:actor:nextMSH"] | ancestor-or-self::node()[@SOAP-ENV:actor="http://schemas.xmlsoap.org/soap/actor/next"])</XPath>
			</Transform>
			<Transform Algorithm="http://www.w3.org/TR/2001/REC-xml-c14n-20010315"/>
		</Transforms>
		<DigestMethod Algorithm="http://www.w3.org/2000/09/xmldsig#sha1"/>
		<DigestValue>wIZFJZjZ7LjKHT6cRmFvlQrxcXA=</DigestValue>
	</Reference>
	<Reference URI="cid:ref1">
		<DigestMethod Algorithm="http://www.w3.org/2000/09/xmldsig#sha1"/>
		<DigestValue>tzvUabyaqBNxxhQ+NfkgpwqTFvk=</DigestValue>
	</Reference>
</SignedInfo>
...

The xml over is generated signing with xmlsec library (https://www.aleksey.com/xmlsec/).
It can also be accomplished with Microsoft's SignedXml class by supplying a resolver to supply the data (Even if Microsoft default has disabled detached signatures).
In this way the signed xml is signing both the envelope and the external payload.
(The external reference could even be an uri like "http://somehost/somefolder/somefile.png")
I cannot see how
signed_root = XMLSigner(method=signxml.methods.detached).sign(root, key=key, cert=cert)

is able to solve this scenario as long as there is no way for the sign method to resolve the uris passed in with the reference_uri parameter. I guess you can reference another node in the xml, but there is currently no way for the signing process to get hold of the external data like in my example.
Regards Asbjørn Oskal

OK, so to clarify, you're looking to be able to generate a detached signature with multiple references, right? That use case is not currently supported.

I must confess I don't understand what <Reference URI=""> means either; can you point me to any docs that describe this construction?

Hi again and thnk you for the quick answer.
Yes you are correct. I need to sign with two references. The first reference which is to an element in the xml itself (ref="") and and the other reference to some external data.
From the specification:
URI="" Identifies the node-set (minus any comment nodes) of the XML resource containing the signature
Or in other words it refers to the element in which the signature element is placed.
Regards Asbjørn Oskal

OK, this should not be very hard to implement, but I'd like to have a complete test example to work with. Can you please supply or find one?

I will try to come up with some suitable example. I guess you mean some code and example xml and payload.
Regards Asbjørn Oskal

XMLDSIGTest.zip

Thanks, I'm taking a look at implementing this.

See https://www.w3.org/TR/xmldsig-core2/#sec-o-SignatureProperty

Hi
I am not sure in what way the link you posted is relevant to my case other than that in
xmldsig 2.0 there are other ways of referencing the objects signed.
The case I have described is what is described in the chapter "Compatibility mode"
https://www.w3.org/TR/xmldsig-core2/#sec-Compatibility-Mode
Here the reference uri like in my case points out of the xml to some content.
In the example it points to "http://www.w3.org/TR/2000/REC-xhtml1-20000126/" so it in
effect signs the HTML content of that link.
Regards Asbjørn Oskal

I posted the link for my own (implementer's) reference as it demonstrates the use of multiple references (one internal and one external). SignXML works fine with external references, it just doesn't yet support multiple references when generating an external reference. That's what this issue is about.

Hi again
You wrote that "SignXML works fine with external references".
Can you give me an example of this using SignXML?
Maybe we mean different things by "external references".
Regards Asbjørn

I gave an example in my first comment: https://github.com/XML-Security/signxml/issues/149#issuecomment-640325908

Hi again
I am sorry but unless you can show me how to reference an external file I do not agree that SignXML does support signing external files like in the c# example code I sent you.
In the SignXml code, all references are run through the _resolve_reference method.
But the only references resolved are empty uris and uris starting with "#".
(And # is added by default to all URIs not starting with # in _get_c14n_inputs_from_reference_uris before calling _resolve_reference)
All calls to _resolve_reference when signing, is done with an uri_resolver method parameter set to None.
So in my case sending in an custom URI like "cid:ref1" would, as far as I can see, first end up like "#cid:ref1". Then this would then be tried resolved by an xpath query which is not what I would want is this case.
What I would want is that a uri_resolver method implemented in my code could instead be called on this "cid:ref1" uri to return the data my code relates to "cif:ref1".
This is what other implementations I have used does.
I still cannot see that SignXML is able to do this.
Regards Asbjørn

Have you tried reading the example in full and following its instructions?
from lxml import etree
from signxml import XMLSigner, XMLVerifier, methods

data_to_sign = "<Test/>"
cert = open("example.pem").read()
key = open("example.key").read()
root = etree.fromstring(data_to_sign)
signed_root = XMLSigner(method=methods.detached).sign(root, key=key, cert=cert, reference_uri="cid:ref1")
print(etree.tostring(signed_root, pretty_print=True).decode())

Here is an example of SignXML performing a detached signature specifying an arbitrary external reference string. The thing missing to support your use case is signing multiple references, including a mix of internal and external ones.
from lxml import etree
from signxml import XMLSigner, XMLVerifier, methods

data_to_sign = "<Test/>"
cert = open("example.pem").read()
key = open("example.key").read()
root = etree.fromstring(data_to_sign)
signed_root = XMLSigner(method=methods.detached).sign(root, key=key, cert=cert, reference_uri="cid:ref1")
print(etree.tostring(signed_root, pretty_print=True).decode())

Hi
I am sorry, but I have been wrong and called my example a detached signature when it in fact is enveloped signatures referencing both an internal and an external source. I guess I was confused by something in the standard and I apologize for that.
Anyway, I ran the example code and I can see that it is possible to generate a detached signature signing an arbitrary xml. Again I'm sorry that we have been talking passed each other.
Then you wrote:
"The thing missing to support your use case is signing multiple references, including a mix of internal and external ones"
I would rewrite this slightly:
"The thing missing to support your use case is signing multiple references enveloped, including a mix of internal and external ones,
where the external ones could point to any binary source".
The standard alos says that URIs without a #-prefix must point to octet streams and is not to be parsed only digested.
("Unless the URI-Reference is such a 'same-document' reference , the result of dereferencing the URI-Reference must be an octet stream")
Then how to refer to external sources?
The standard only mentions http URIs explicitly:
"We RECOMMEND XML Signature applications be able to dereference URIs in the HTTP scheme"
However the three implementations I have used (xmlsec, microsoft and java) all supports other custom protocols (like cid: in my case) by supporting custom resolvers.
Regards Asbjørn Oskal

I may be rusty on my spec but I don't see a fundamental difference between an enveloped signature with external references outside the enveloping document and a detached signature with external references. You can generate a detached signature and then embed it in a document, making it enveloped.
But you raise a good point - SignXML needs to be able to handle string (octet-stream) references for detached signing, not requiring them to be XML nodes. I've updated the title of the issue to reflect that.

I have never tried it, but I guess you are right about embedding a detached signature in a document to make it enveloped.
Sorry again for a lengthy discussion. It improved my spec knowledge which also may have been a bit corroded.
Regards Asbjørn

Do we have any update on signing / handling data other than XML nodes in the external references? I am currently trying to work this out as well since we need to secure the soap attachment during the request conforming to the SwA standard (http://docs.oasis-open.org/wss-m/wss/v1.1.1/os/wss-SwAProfile-v1.1.1-os.html).

Also, I have worked some prototypes to build a list of reference:
[{id = cid: foo, data = data}, {id = bar, data = data}, etc.]
and append this to reference_uri, and then in the back then when processing the reference_uri, detect whether the iterator is a list and hence process the id and data accordingly.
One question I have been trying to figure out is how to pass the gzip.file into the c14input and what should be the c14 output as the digest input. Since the data is not a xml node then c14n is not applicable per the standard. Would be great to hear your thoughts.
Best regards
Isaac

The outcome of the discussion above was that signxml currently lacks the capability to sign multiple external references, and also lacks the capability to sign external references that are not XML nodes.
In all likelihood, the only type of data that will be accepted for signing instead of XML nodes will be an arbitrary bytestring or bytestream. I don't expect there will be special handling for any other data types.
I maintain this project in my spare time; there is no ETA for when this feature will become available. PRs and sponsorships are welcome.

Thank you for your reply. I have been trying out this new signing for the attachment over the weekend and the server has been responding 'signatures verifications error'. The file I have sent through is a gzip file and contain arbitrary binary data and hence no transform has been done. The data is inputted directly as pre-digest value and hence go through the digest value accordingly and successfully obtain the reference node and hash values and etc. the binary data is attachment in the MIME content without any transfer encoding with the email.message.MIMEApplication. Would you be able to share some thoughts on how to troubleshoot? I would reckon once this is settled, my code will be able to assist the project to develope this features. Much appreciated for your time on this. Regards, Isaac

Hi there,
Hope you have been well. This is a great project indeed! I cannot get enough of it but the non-XML content signing part gives me no choice to use other coding languages in the back end. I still prefer python for the backend.
Just wondering would you be able to support the signing of non-XML content? If there anything I can help to fast track this feature?
thanks & regards
Isaac

Thanks. There are two ways you can help: by donating your time to write a quality PR for that feature, or donating money to support this project using the "Sponsor" button at the top of the page.


Updated UI for the Options dialog

This should fix #444
Also changed the default for the checkbox for override proxy settings to unchecked.
UI now looks like this

:white_check_mark: Build ServiceBusExplorer 4.2.0-FixOptionsWindowSize.65.build.697 completed (commit https://github.com/paolosalvatori/ServiceBusExplorer/commit/deb5af34ec by @ErikMogensen)


Closing the loop: listsamplesize group results for the Mockingbird-N1 system.

Adding the missing results for listsamplesize language group in the development set.

Thanks!


Fix #1976 name of the user is visible in navigationView

Fixes #1976
Changes:If the user logs in and changes his/her name in the editProfileActivity, that name will be displayed in the navigationView instead of his email
Screenshots for the change:

@heysadboy done with the requested changes

@nikit19 Please resolve conflicts

@iamareebjamal done with the requested changes

@nikit19 Please resolve conflicts

conflicts are resolved


Ends at missing python

Script
nginx-proxy-manager
sh -c "$(wget --no-cache -qO- https://raw.githubusercontent.com/ej52/proxmox/main/install.sh)" -s --app nginx-proxy-manager
Describe the bug
Errors and quits the installer when checking for python.
Python3 and python3-pip is installed.
✔ Operating System Updated
✔ Dependencies Installed
✔ Rust v1.74.0 Installed
✘ Python not Installed
E: Sub-process /usr/share/proxmox-ve/pve-apt-hook returned an error code (1)
E: Failure running script /usr/share/proxmox-ve/pve-apt-hook
[ERROR] on line 99 with exit code 100 while executing command pkg_del
System info:

Proxmox Version: 8.1.4
Alpine Version: 3.18

Hi is this Inside a CT ? A new install and Template this:

?
Never seen this before --> E: Failure running script /usr/share/proxmox-ve/pve-apt-hook
This is more a indicator of an Bad upgrade
KR

This error comes both from shell and ssh when trying to use the script to install NPM.
So it does not even let me start an CT...

This is from the Proxmox Host, not inside the CT.
Last time I used it, it worked directly from the host.

exactly ;D

https://github.com/vontainment/proxmox-scripts
I fixed all the errors in my repo and made a pull request until its accepted you can use my repo.

this is for install on alpine, ubuntu or debian. ct, vm baremetal. not though proxmox


No bindings for inline class functions

This works:
// cpp/Test.hpp
#pragma once

class Test {
private:
    int m_counter;
public:
    Test(int);
    int incr();
};

// cpp/Test.cpp
#include "Test.hpp"

Test::Test(int c) : m_counter(c) {}
int Test::incr() { return m_counter++; }

But this doesn't:
// cpp/Test.hpp
# pragma once

class Test {
private:
    int m_counter;
public:
    Test(int c): m_counter(c) {};
    int incr() { return m_counter++; };
};

Neither new nor incr methods are generated.
Generated code:
/* automatically generated by rust-bindgen */

pub use self::root::*;

#[allow(non_snake_case, non_camel_case_types, non_upper_case_globals)]
pub mod root {
    #[allow(unused_imports)]
    use self::super::root;
    #[repr(C)]
    #[derive(Debug, Copy)]
    pub struct Test {
        pub m_counter: ::std::os::raw::c_int,
    }
    #[test]
    fn bindgen_test_layout_Test() {
        assert_eq!(::std::mem::size_of::<Test>() , 4usize , concat ! (
                   "Size of: " , stringify ! ( Test ) ));
        assert_eq! (::std::mem::align_of::<Test>() , 4usize , concat ! (
                    "Alignment of " , stringify ! ( Test ) ));
        assert_eq! (unsafe {
                    & ( * ( 0 as * const Test ) ) . m_counter as * const _ as
                    usize } , 0usize , concat ! (
                    "Alignment of field: " , stringify ! ( Test ) , "::" ,
                    stringify ! ( m_counter ) ));
    }
    extern "C" {
        #[link_name = "_ZN4TestC1Ei"]
        pub fn Test_Test(this: *mut root::Test, arg1: ::std::os::raw::c_int);
    }
    impl Clone for Test {
        fn clone(&self) -> Self { *self }
    }
    impl Test {
        #[inline]
        pub unsafe fn new(arg1: ::std::os::raw::c_int) -> Self {
            let mut __bindgen_tmp = ::std::mem::uninitialized();
            Test_Test(&mut __bindgen_tmp, arg1);
            __bindgen_tmp
        }
    }
}

build.rs
extern crate bindgen;
extern crate gcc;

use std::env;
use std::path::PathBuf;
use bindgen::Builder;

fn main() {
    gcc::Config::new()
        .cpp(true)
        .file("cpp/Test.cpp")
        .compile("libtest.a");

    let bindings = Builder::default()
        .enable_cxx_namespaces()
        .raw_line("pub use self::root::*;")
        .header("cpp/Test.hpp")
        .clang_arg("-x")
        .clang_arg("c++")
        .clang_arg("-std=c++11")
        .generate()
        .expect("Unable to generate bindings");

    let out_path = PathBuf::from(env::var("OUT_DIR").unwrap());
    bindings
        .write_to_file(out_path.join("test.rs"))
        .expect("Couldn't write bindings!");
}

Note that out-of-line definitions in the header also work:
// cpp/Test.hpp
#pragma once

class Test {
private:
    int m_counter;
public:
    Test(int);
    int incr() { return m_counter++; };
    int decr();
};

Test::Test(int c): m_counter(c) {};

int Test::decr() {
    return --m_counter;
}

The above generates new and decr but not incr.

This is expected, inline functions don't generate symbols, so there's no way we can call into them.
You can compile with -fno-inline-functions, I guess, but that would destroy the perf of your library I suppose.

This is expected, inline functions don't generate symbols, so there's no way we can call into them.

There can be symbols (name mangling doesn't require that the thing whose name being mangled is not inline, and the name can appear, for example, in the DWARF). However there isn't a non-inlined function we can link to0.
0 There may be situations you can trick the compiler into creating such a canonical, non-inlined version of the function. Taking the function pointer of an inline function should do it. But yes, it would destroy performance, just as -fno-inline-functions will.

You can compile with -fno-inline-functions, I guess, but that would destroy the perf of your library I suppose.

Additionally, this wouldn't work with system libraries or any library that you aren't compiling yourself.


[Text Analytics] Log warn-text response header

Resolves: https://github.com/Azure/azure-sdk-for-net/issues/33421

API change check
API changes are not detected in this pull request.


Azure backup model validations are failing after tool update

Fix the failures in this PR - https://github.com/Azure/azure-rest-api-specs/pull/6818

@ArcturusZhang please assign this issue to me.


Feedback: Access Azure Data Lake Storage gen1 using Microsoft Entra ID credential passthrough in Azure Databricks

Document Correction Proposal: Access Azure Data Lake Storage directly using credential passthrough in Azure Data Lake Storage Gen1.
Documentation: https://learn.microsoft.com/en-us/azure/databricks/archive/credential-passthrough/adls-passthrough
Overview:
As per the Documentataion we can access the data from Azure Data Lake Storage Gen 1(ADLS Gen1) but this service is not supported now in azure it is discontinued from 29 feb 2024.

Still document is not updated with this information.
User can not be able to access data from ADLS gen1 First user need to Migrate data from ADLS Gen1 to ADLS Gen 2. You can refer this MS document for more information on migration.
After migrating the data to ADLS Gen2 You can access it from Azure Databricks.

@PratikAnkushLad
Thanks for your feedback! We will investigate and update as appropriate.

@PratikAnkushLad
Thanks for your feedback! I've assigned this issue to the author who will investigate and update as appropriate.

@pesalapavan this is not the correct repo. Azure Databricks content does not use github issues for feedback, and is not part of azure-docs.
FYI @whhender  some Archive content is still live I guess?

@PratikAnkushLad  the top of the page says "This documentation has been retired and might not be updated." since the page has been Archived and is no longer updated.  Sorry for any confusion it caused.


's character counter does not disappear after removing maxlength attribute but show 0 instead

What is affected?
Component
Description
Use either md-outlined-text-field or md-filled-text-field, set the attribute maxlength to any valid value, then remove it.
The character counter now shows something like 5/0 instead:

Reproduction
See above. A live check can be seen here: https://blazorwebmaterial.lukevo.com/text-field by checking then unchecking the Max Length option.
Workaround
None. The entire element has to be scraped and a new one created. Not an option for frameworks I think.
Update maxlength attribute to -1 instead of 0 or removing the attribute.
Is this a regression?
No or unsure. This never worked, or I haven't tried before.
Affected versions
1.0.0-pre.17 and the code built on Sep 20th
Browser/OS/Node environment
Microsoft Edge Version 117.0.2045.41 (Official build) (64-bit)
Other info is N/A

oops 😬


ensureScreenSize non-recursive version ?

I have same issue https://github.com/OpenFlutter/flutter_screenutil/issues/402
It seem have concern about recursive issue for stackoverflow.
Is it possiable provide ensureScreenSize non-recursive version ?

Please try this first


tests/rendering/depth: Key.minus and Key.equals values in Keycodes.hx don't match up to reality

Looking through the tests and demos I came across the rendering/depth test. On a MacBook I am getting the following when tracing e.keyCode in the onkeyup method:
'=' is 187
'-' is 189
Key.equals is defined as 61
Key.minus is defined as 45
Also, I assume the 'if' statement checking for Key.space wants to reset the depth to 0 when right now it just sets the depth to itself which doesn't do anything. The contents of the if block should be:
sprites[0].depth = 0;

Note: once I figure out how to get a Git pull of Luxe working I'll submit proper pull requests for code changes. My apologies for submitting the last part of this in a bug report.

I can only guess you were you running on the web target? What browser + version, language locale, and OS were you running if so?

Web target. That is correct. Tested in Safari 9.0 and Chrome 46.0.2490.33 beta. Below is a dump of a request Chrome with locale/OS info:
Accept-Language: en-US,en;q=0.8
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2490.33 Safari/537.36


Dedicated Open Graph Images

Context
For best SEO and social media sharing, each page should have a dedicated image just for this purpose. This is the og:image in Open Graph protocol. Please design the OG images for each page with the following specs:

represent the page with minimal details,
in light mode
aspect ratio 1200:630
2 variants for EN & VI

Ref: https://developers.facebook.com/docs/sharing/webmasters/images/
### Tasks
- [x] Fallback OG image
- [x] Dedicated OG image for home page
- [x] Dedicated OG image for events page
- [x] Dedicated OG image for blog page
- [x] Dedicated OG image for roadmap page
- [x] Dedicated OG image for coc page

Done via https://github.com/sveltevietnam/sveltevietnam.dev/commit/7a2d2c8be69656a70eac36d4f4cd4843f4369e34


Uses for Aquarium and Star Steel

Maybe aquarium can be used for EnderIO's enchanter to make depth strider/respiration/aqua affinity books. I don't know what to do for star steel though.

An alternate material for the mending enchantment, since Star Steel self repairs?

Also, aqua affinity takes a lilypad. Depth strider takes prismarine. Respiration takes empty bottles. Aquarium blend takes prismarine, so everything here is cheaper than aquarium ingots. Do you want me to replace a cheaper recipe with a more expensive one just to use aquarium?
I wouldn't mind using Star Steel for some late game magical machines or something. I'll have to think about it though.

make Aquarium take less xp then other materials in the enchanter to get those enchantments
Aquarium alternate recipies for evilcraft's infinite water block, that magic chalice thing from reliquary,
Star Steel dust + 8 iron dust make 9 steel dust.
Star Steel Gear for Essence Refiner, and maybe the rocket assembler?
Once we get the AM update we should be able to soon add aquarium and starsteel to the spell crafting altar.

Respiration seems way too cheap

Maybe from 8 levels to 4

Needs to be tested

Having multiple recipes for the same enchant is not implemented properly. Should the altered recipes be kept? @Divineaspect


caskroom capitalization

This is very minor, but when running the macos-install.sh script on my Mac, I had to change
brew tap caskroom/cask
to
brew tap Caskroom/cask
in the XQuartz section to get the install to work.  I'm not sure if this is a recent brew change.

Thanks for the report. Would you be willing to send a quick patch?

Sure, though I'm not sure if this affects other people.  It would be good if some other mac user could try a clean install and see if this is a unique problem.

What version of OS X are you on? I ran into issues installing on my Mac but I can't recall exactly what they were.

10.11.  Again not sure if this affects others, though I found mention of a similar problem in other projects.

Patched has been merged.


Deprecation calls need to be updated for compat with Ember 3.24

These errors are triggered before deprecation-workflow can intercept them
Error: When calling `deprecate` you must provide `for` in options. Missing options.for in "ember-cli-page-object.old-collection-api" deprecation
Error: When calling `deprecate` you must provide `since` in options. Missing options.since in "ember-cli-page-object.old-collection-api" deprecation
Error: When calling `deprecate` you must provide `for` in options. Missing options.for in "ember-cli-page-object.create-url-argument" deprecation
Error: When calling `deprecate` you must provide `since` in options. Missing options.since in "ember-cli-page-object.create-url-argument" deprecation
Error: When calling `deprecate` you must provide `for` in options. Missing options.for in "ember-cli-page-object.string-properties-on-definition" deprecation
Error: When calling `deprecate` you must provide `since` in options. Missing options.since in "ember-cli-page-object.string-properties-on-definition" deprecation

Scratch that, these are being thrown from ember-source deprecations: ember-source.deprecation-without-for
and I have throwOnUnhandled: true in deprecation workflow.

According to the https://github.com/san650/ember-cli-page-object/issues/512, I think we should replace deprecate coming from the '@ember/application/deprecations', with our home grown deprecate util(or some 3rd party standalone util). It may also support the for field. I think this should be done for both v1 and v2, since there should be no other blockers to use v1 with the latest version of ember.

According to the https://github.com/san650/ember-cli-page-object/issues/512, I think we should replace deprecate coming from the '@ember/application/deprecations', with our home grown deprecate util(or some 3rd party standalone util). It may also support the for field. I think this should be done for both v1 and v2, since there should be no other blockers to use v1 with the latest version of ember.

Alright, I just realized it's not about the point when deprecations are being fired, it's about any time ember's deprecate( is invoked... I'll take a look at it.


Handle URL's for images if URL is given instead of a local fs path.

Context:
In php's version of fpdf, we are able to use a http URL as argument to fpdf.Image() like
// Insert a dynamic image from a URL
$pdf->Image('http://chart.googleapis.com/chart?cht=p3&chd=t:60,40&chs=250x100&chl=Hello|World',60,30,90,0,'PNG');

Currently, we only can read from the local filesystem using os.Open.
Summary
This commit checks if the path fileStr passed to RegisterImageOptions is a http URL, and if so, attempts to fetch the image. It otherwise defaults to the usual behaviour of reading via os.Open.

@jung-kurt , kindly review.
Also thanks for this library. Our team relies on it heavily.


[issue] this.items.length is not a function

I tried to use this very interresting plugin to paginate an  list but I encountered this error:
this.items.length is not a function
I use jquery 3.1.0

I tried to replace .size by .length but same issue

I think this should be this.items.length and not this.items.length()

Hi @nutzmdm, thanks for pointing this out. You are correct! The documentation says .size() is deprecated since jQuery 1.8.
Oops! 😬
I will fix this in a few minutes.


CASSGO-7 Change Batch API to be consistent with Query()

According to issue #1187, I have refactored the Query() method for the batch and implemented the Exec() method for the batch structure to save the general pattern of the query execution, and saved the older method (session.ExecuteBatch()) for backward compatibility.
This means the batch for now behaves the same way as query.

@tengu-alt we should also add a new Batch() function and deprecate the existing NewBatch (basically a "soft" rename of NewBatch() -> Batch() without breaking users) and deprecate Session.ExecuteBatch.
NewBatch and ExecuteBatch can then be removed in 3.0 so that users have more than enough time to transition to the new names.

@tengu-alt we should also add a new Batch() function and deprecate the existing NewBatch (basically a "soft" rename of NewBatch() -> Batch() without breaking users) and deprecate Session.ExecuteBatch.
NewBatch and ExecuteBatch can then be removed in 3.0 so that users have more than enough time to transition to the new names.

@joao-r-reis I do not clearly understand. The NewBatch() function is already marked as deprecated and session.NewBatch  is recommended to use instead. Also I can't name the function same as struct (Batch) because the Batch is already declared as struct.

Session.NewBatch() should be deprecated now, global NewBatch() can be removed since it was deprecated already and Session.Batch() should be added to replace Session.NewBatch() (keep the same functionality). Session.Batch() does not have a conflict with the type Batch, you can have both (in fact this already exists with Query() which is what I'm suggesting we do for Batch as well.

global NewBatch() removal is already handled by https://github.com/apache/cassandra-gocql-driver/pull/1762 so it doesn't have to happen here

Session.NewBatch() should be deprecated now, global NewBatch() can be removed since it was deprecated already and Session.Batch() should be added to replace Session.NewBatch() (keep the same functionality).
Session.Batch() does not have a conflict with the type Batch, you can have both (in fact this already exists with Query())
The original issue mentions the rename Session.NewBatch() to Session.Batch() already, my only additional suggestion here is to keep and deprecate Session.NewBatch() for now AND add the new Session.Batch().

I see, thank you. Done, also I've changed the Session.NewBatch() usage in .doc and tests.

Sorry I merged another PR which is causing a git conflict on CHANGELOG.md can you fix it and I'll merge this right after? Otherwise I can merge manually and fix it in place

Sorry I merged another PR which is causing a git conflict on CHANGELOG.md can you fix it and I'll merge this right after? Otherwise I can merge manually and fix it in place

Fixed! Thank you for the heads up.


feat(http-client): Forward Requests from response interceptors

If a response interceptor returns a new Request, the fetch client resolves a fetch call to that Request.

@AshleyGrant Here is a PR that implements the feature originally presumed by @CasiOo. You can forward a Request from a response interceptor and it will rerun the fetch. This makes refetching trivial (see https://github.com/aurelia/fetch-client/pull/105/files#diff-69d1ba4b313b5f4fc7ca97fff2bf4ae8R273).
The upside is that this adds some powerful options including but not limited to retry. It also makes adding retry as a first class citizen as simple as a 5 line interceptor and the configuration code fromhttps://github.com/aurelia/fetch-client/pull/104.
The downside is that response interceptors probably expect responses. I doubt this is going to be a breaking change for anyone, as if a Response is returned the developer is likely responsible. However, it might be prudent to add a distinct implementation for processResponse that aborts immediately in the event that a Response is encountered.
Feedback welcome. @eriklieben @EisenbergEffect

Finally, this code doesn't actually allow us to fully implement the RetryInterceptor with the beforeRetry callback "in the box", as the interceptor still does not have access to the HttpClient instance, which is key to the use case of allowing the developer to possibly update the configuration of the HttpClient before doing the retry.

I've added the HttpClient as a new argument to the interceptors.


Unchain function nodes in backward

Currently, all the function nodes in the computational graph is kept alive until whole backward computation is finished. Essentially, function nodes whose gradients are already propagated to all the input nodes are no longer needed and can be freed. If such function node holds large arrays as its attributes (usually in order to save common calculation between forward() and backward()) it unnecessarily occupies the memory.
This PR is to free such function nodes.
There are two types of references to such function nodes:

References from output variables' creator property.
seen_set in the backward graph traversal.

For 1., creator nodes of each variables are unchained as traversing backward.
For 2., weak references is used in seen_set.
Code:
This is somewhat artificial example, but can demonstrate the effect.
import chainer
from chainer.backends import cuda
import cupy

class FooFunc(chainer.FunctionNode):
    def __init__(self, name):
        super().__init__()
        self.name = name

    def __del__(self):
        self.bar = None
        print("{:10d} DEL: {}".format(cuda.memory_pool.used_bytes(), self.name))

    def forward(self, inputs):
        # Dummy array which can be used in backward()
        self.bar = cupy.ones((100, 100, 100, 10), cupy.float32)
        x, = inputs
        y = x + 1
        print("{:10d} FWD: {}".format(cuda.memory_pool.used_bytes(), self.name))
        return y,

    def backward(self, indexes, grad_outputs):
        gy, = grad_outputs
        print("{:10d} BWD: {}".format(cuda.memory_pool.used_bytes(), self.name))
        return gy,

def func(x):
    hs = x,
    hs = FooFunc('a').apply(hs)
    hs = FooFunc('b').apply(hs)
    hs = FooFunc('c').apply(hs)
    hs = FooFunc('d').apply(hs)
    hs = FooFunc('e').apply(hs)
    h, = hs
    return h

def main():
    x = chainer.Variable(cupy.ones((2,)))
    y = func(x)
    y.grad = cupy.ones_like(y.data)
    y.backward()
    print(y, y.grad, x.grad)

main()

Result (master):
  40001024 FWD: a
  80001536 FWD: b
 120001536 FWD: c
 160001536 FWD: d
 200001536 FWD: e
 200001536 BWD: e
 200001536 BWD: d
 200001536 BWD: c
 200001536 BWD: b
 200001536 BWD: a
variable([ 6.  6.]) [ 1.  1.] [ 1.  1.]
 160000512 DEL: e
 120000512 DEL: d
  80000512 DEL: c
  40000512 DEL: b
       512 DEL: a

Result (This PR):
  40001024 FWD: a
  80001536 FWD: b
 120001536 FWD: c
 160001536 FWD: d
 200001536 FWD: e
 200001536 BWD: e
 160001536 DEL: e
 160001536 BWD: d
 120001536 DEL: d
 120001536 BWD: c
  80001536 DEL: c
  80001536 BWD: b
  40001536 DEL: b
  40001536 BWD: a
      1536 DEL: a
variable([ 6.  6.]) [ 1.  1.] [ 1.  1.]

It breaks backward compatibility because repeated call of backward() will be ignored.
x = chainer.Variable(numpy.ones((1,), numpy.float32))
y = x * x + x
y.backward(); print(x.grad)
y.backward(); print(x.grad)
y.backward(); print(x.grad)

master:
[ 3.]
[ 6.]
[ 9.]

This PR:
[ 3.]
[ 3.]
[ 3.]

So it may be better to make it configurable, but I wonder if this master's behavior is documented (at least it's not tested, because the unit test is successfull (except irrelevant doctest code)).

Maybe it's not documented, but we should be careful to change such a fundamental behavior. I think it is well known (at least among old users) that multiple calls of backward accumulate the gradients, so there may exist user code depending on this behavior.

Should this PR be further discussed (maybe raise an issue to enhance the docs) or closed for the time being? I also think that user can benefit from this feature but agree with @beam2d .

I think this PR needs not to be closed.
Backward compatibility can be kept by making the behavior configurable.
But I think it's better to change the default behavior to unchaining (thus no-compat), but issue an error if user tries to call backward() twice without disabling this behavior.
That would benefit most of users, and still avoid silently breaking existing user code.

Then let's leave it as is and continue with the discussion. I'd personally like to see this feature but maybe we should ask the others too.

How about moving the core of Variable.backward() to FunctionNode.backward(), which would allow users to write the following?
func = y.creator_node
del y
func.backward()

I confirmed this approach works:
    cont = y.backward(execute=False)
    del y
    cont()

results
$ python test4301.py
  40001024 FWD: a
  80001536 FWD: b
 120001536 FWD: c
 160001536 FWD: d
 200001536 FWD: e
variable([1. 1.]) variable([32. 32.])
 200001024 BWD: e
 160001536 DEL: e
 160001536 BWD: d
 120002048 DEL: d
 120001536 BWD: c
  80002048 DEL: c
  80001536 BWD: b
  40002048 DEL: b
  40001536 BWD: a
      2048 DEL: a
variable([32. 32.]) variable([1. 1.])
[variable([32., 32.])]

(code: https://github.com/chainer/chainer/compare/master...toslunar:c534a0964e9e2e812eba2ab3301b1862651cd4ad)
However it required some refactoring like grads.pop.  It's welcome to take over the code.  If no one does, I'll work on this issue after #4772 is merged.

After an internal discussion, we concluded to close the PR for now.

It doesn't improve total memory consupmtion; it's only beneficial when large memory is consumed during backward computation. Currently we haven't come up with such case.
It may not be worth introducing another configurable parameter that can potentially break the exising code.
We may revisit the PR when we recognize a case where this fix is beneficial.

Please feel free to reopen ths PR if anyone wants this feature.


Add secrets to config schema

This PR adds secrets to our posit-publishing-schema-v3.json for Configuration files.
It is an array of strings to indicate the name (not value) of secrets that are available to be published alongside a deployment:
secrets = ['API_KEY', 'DATA_PASSWORD']

Intent
Resolves #2251
Type of Change

[ ] Bug Fix

[x] New Feature

[ ] Breaking Change

[ ] Documentation

[ ] Refactor

[ ] Tooling

Approach
~This doesn't include changes to the Content Record, but will be added in a future PR.~
After a bit more work on this I realized that isn't necessary.
A lot of file changes you may expect are missing from this PR since secrets was already in our Frontend and Backend types for Configuration details. We just didn't specify them in the schema.
I did not increase the schema version for this since older schemas are not made invalid since this is only adding an attribute.
Automated Tests
No automated tests were added. The secrets attribute was already present, and TOML verification is already being tested.

Going to wait to merge this until I discuss and document how we want to handle schema changes with @mmarchetti
I'll also update the tests

After advisement from @mmarchetti the schema was updated on the CDN after this was merged.

I verified that I can add secrets to the config but it's not clear how to populate those in Connect.  I assumed that I just need to have an environment variable with the name.  For example, I have an environment variable named DOGFOOD_API_KEY. So I assumed adding this to my config would create an Environment Variable in Connect:
secrets = ["DOGFOOD_API_KEY"]
However it didn't.  How do I actually use the secrets?  Or has that not yet been implemented?

After advisement from @mmarchetti the schema was updated on the CDN after this was merged.

Is this documented somewhere? Should it be?

Is this documented somewhere? Should it be?

Documentation has been added in #2279

However it didn't. How do I actually use the secrets? Or has that not yet been implemented?

Nothing has been implemented yet. This is just the first step, allowing them to be declared in the Configuration. #2273 gives us the API the ability to take Secrets and set them. There is no PR yet for how to set them in the extension, but I should have a PR/demo to present soon!

Thanks for the clarification!


utils: drop cmake version check

There is no guarantee that cmake is available in the path. CMake may be provided through VS or the VS Build Tools, in which case, you must be in a VSDevCmd environment to invoke cmake. This adds quite a bit of complexity for little gain as the projects will check of the minimum version required (cmake_minimum_required).

CC: @weliveindetail @hjyamauchi

@swift-ci please smoke test

Good point. I have no strong opinion on it, but if we remove this check, we might want to raise the cmake_minimum_required version in the top-level swift project from 3.19 to 3.28, because this is what swift-inspect requires https://github.com/swiftlang/swift/blob/ee9317a7b574340c8e7c7855f24592e073c57676/tools/swift-inspect/CMakeLists.txt#L1

@weliveindetail swift-inspect is built as a standalone project, not part of the Swift toolchain build, so the top level CMake version check doesn't impact it.
Yeah, I think that I would rather make things unconditional if possible.

swift-inspect is built as a standalone project, not part of the Swift toolchain build, so the top level CMake version check doesn't impact it

@compnerd Yes, just saying that it's suboptimal to bail out so late in the build process due to an insufficient CMake version. That was the intention of the early check, but yeah it's not critical.


Implemented Linux versions UTF8 <-> ASCII and UTF8 <-> ISO 8859-15, GetLastErrorString

Implemented GetLastErrorString
I've returned string representations of the errno codes used in LinuxUtils.cpp, since the numerical value of the codes differs on most systems so returning the numerical constant would be meaningless.
It would be possible to implement this better using the C++11 error_category feature but that would mean depending on a C++11 compatible libstdc++ and I wasn't sure what C++ standard the project is built against.

Used POSIX iconv to implement a generic conversion function.
Currently, I've tested UTF8 <-> ASCII and UTF8 <-> ISO 8859-15.

The only drawback is that iconv requests a non-const pointer to the input byte sequence so it apparently does not guarantee that the input sequence is not changed. I've implemented the function so that it copies the input of a conversion function beforehand. In practice, it doesn't seem to change the input, so it might be allowable to const_cast the original string buffer to avoid the copy but I can't guarantee this will work on all environments without further testing.
If using iconv is acceptable I'll implement the remaining UTF-16 conversions and do a few tests with complete savegames to test all Linux specific functions are working.
I'm not really sure exactly what FromMultiByte and ToMultiByte should return, so I've not touched them yet.

I also assume that each wchar_t in the UTF-16 conversion output should be a single 16 bit part of an UTF-16 character and not a character in the system dependent wchar_t charset. If the former, the conversion function needs to convert to UTF-16 and then create a wchar_t out of every 2-4 bytes in the output from iconv (using little endianness, since that is the default in Windows). If the former, I could use iconv's conversion algorithm to directly convert it into (system dependant, probably unicode codepoints) wchar_t values but then the output would not be a UTF-16 character string.

I've looked over the usage, and the strings passed to the conversion functions should not be modified.
FromMultiByte and ToMultiByte seem to be remnants of an initial attempt at implementing UTF8 support, and are not needed.
UTF-16 is used in two ways: for interacting with the Windows filesystem (which you won't care about), and as an intermediate representation of 8859-15 files so they can be parsed. AFAIK it is actually UTF-16, but that's dependent on Windows being honest about the character conversions it's doing. As UTF-16 never ends up being used externally, I say go with the easy method and see where it gets you in testing. If the parser chokes, you know to use the more difficult method.

OK, I've tested it:

Converting to UTF-16 LE and storing the result in a wchar_t string is possible but some normal string methods no longer work because char_traits<wchar_t> on my system assumes that the data is encoded in UTF-8. It works on Windows because UTF-16 LE is the system dependent wchar_t locale.
If I convert it using iconv's default "wchar_t" encoding it turns out as UTF-8, which is consistent with my system's encoding. Parsing saves seems to work, but I've not yet managed a successful one (missing mods & files). The only drawback is that the API calls it a conversion to an UTF-16 string (when it is actually UTF-8).

I'm going for option 2 and test until the saves convert successfully, then I'll create another pull request.


rand.c WIN32 code path

I've noticed that you use srand(time(NULL)) before each random32,
this is not a good idea as time() it has a granularity of 1 second (!!!!) so it's very likely
that a sequence of random32 calls generates the same pseudo-random numbers...
Also, using rand() on Windows means naive LCG.
This very unsafe IMHO.
I'm wondering - why not replace it with CryptoAPI's CryptGenRandom?

WIN32 support in trezor-crypto is not a first-class citizen, because the library is meant to be used on embedded microcontrollers such as ARM Cortex-M4.
I am not sure anymore why we added WIN32 support to the library, probably to satisfy some unit tests, but I would not recommend anyone using this library under Windows.
That said, feel free to send a pull request with Win32 fixes if you care about this use case.


support CTE in Nereids

This issue is used to record the progress of the implementation of CTE functions and subtasks in nereids. Currently, the CTE syntax is implemented by fully inlining it(in #12742), without cost-based optimization. Based on approaches and algorithms in paper Optimization of Common Table Expressions in MPP Database Systems, there are still a few more subtasks to be done, which may refactor the current CTE implementation.
Subtasks for this issue:

[x] the completed CTE-related syntax, except keyword RECURSIVE, which is not supported in existing implementation; https://github.com/apache/doris/pull/12742
[ ] data structures (and corresponding logical and physical operators, if necessary) for CTE, includes CTEProducer, CTEConsumer, CTEAnchor, Sequence;
[ ] transformation rules for CTE, includes anchorToNoOp, anchorToSequence, consumerToInlinedCTEPlan;
[ ] algorithm to check and eliminate invalid CTE derived plan(generated by CTE transformation rules);
[ ] optimization for CTEConsumer, includes predicate push down(specific to CTE), inline single-used CTEs, eliminate unused CTEs recursively;
[ ] choose the best CTE plan alternatives by CBO;
[ ] materialize results of CTEProducer, and deliver those results to corresponding CTEConsumer in correct order(to avoid deadlocks);

This issue is used to record the progress of the implementation of CTE functions and subtasks in nereids. Currently, the CTE syntax is implemented by fully inlining it(in #12742), without cost-based optimization. Based on approaches and algorithms in paper Optimization of Common Table Expressions in MPP Database Systems, there are still a few more subtasks to be done, which may refactor the current CTE implementation.
Subtasks for this issue:

[ ] the completed CTE-related syntax, except keyword RECURSIVE, which is not supported in existing implementation; https://github.com/apache/doris/pull/12742
[ ] data structures (and corresponding logical and physical operators, if necessary) for CTE, includes CTEProducer, CTEConsumer, CTEAnchor, Sequence;
[ ] transformation rules for CTE, includes anchorToNoOp, anchorToSequence, consumerToInlinedCTEPlan;
[ ] algorithm to check and eliminate invalid CTE derived plan(generated by CTE transformation rules);
[ ] optimization for CTEConsumer, includes predicate push down(specific to CTE), inline single-used CTEs, eliminate unused CTEs recursively;
[ ] choose the best CTE plan alternatives by CBO;
[ ] materialize results of CTEProducer, and deliver those results to corresponding CTEConsumer in correct order(to avoid deadlocks);

相关论文阅读文档
https://q8g7386elf.feishu.cn/docx/DyXcd8vxZoKvTtxqu0icvaZbnYe

Hello, Doris' developer. I am very interested in optimizing  CTE. And I have some ideas on the materialization of CTE.May I join  you？ @morrySnow  @YangShaw


Add import/export eslint plugin

https://www.npmjs.com/package/eslint-plugin-import

Fixed by https://github.com/bnjbvr/kresus/commit/15ab09dc4ea87f36087a2228e2af25803d34ef6e


Improved seed data

Would be great to have a way to insert useful data into database. Slightly complicated by the database being shared with the foodsharing desktop site which has it's own data seeding system (manual sql for now).
Looks like DoctrineFixturesBundle [1] and AliceBundle [2] are the nice thing with --append to keep the other data:
./bin/console doctrine:fixtures:load --append

This guide [3] looks reasonable too.
[1] http://symfony.com/doc/current/bundles/DoctrineFixturesBundle/index.html
[2] https://github.com/hautelook/AliceBundle#basic-usage
[3] https://knpuniversity.com/screencast/symfony-doctrine/dummy-data

AliceBundle --append is broken for now, but I think it is totally fine to use it in purge mode (which may need to reinitialize database when working on the desktop site) and just add all fixtures as needed.
I setup an example/.
Missing step would be to integrate alice and factory-muffin to reuse "default fakes" for the entities. I tried doing it by having a Factory Service that registered all factories and made them available - but using that in alice __construct did not work as expected. For now, I am fine without the factories here.

Rethinking this, I will get rid of Alice and add tooling to use factory-muffin together with DoctrineFixturesBundle. Even the codeception documentation notes that factory-muffin is similar to Alice...


Allow to load a single browser window from a remote URL

This tracks making it easy to run test UI hosted on a remote URL.  This could go in a lot of directions, but I'd like to be able to navigate to a known page with a list of git branches hosted on github, and then click each link to open a new window with UI served from each git branch.  That would make it very easy to test PRs and run field tests in the wild.  (Much easier than packaging complete test browsers, which gets unwieldy.)
Technically, there are some challenges.  WebView pre-load scripts must be served from secure origins; in practice, that means they must be file:// URLs.  There is support for https://www.npmjs.com/package/asar built into Electron, which might help.  Or we could fetch entire branches using github's archive endpoint, possibly converting to asar on the fly.

@canuckistani if you want to influence how this looks and feels, or what it enables, please give direction.

This sounds awesome!

I believe this should be duped to https://github.com/mozilla/tofino/issues/905


Add genome mean normalization argument to signal_from_telomeres_average()

It is more useful to visualize signal at sub-telomeric regions when normalized against average genome-wide signal. Add the functionality to the function, under control of a boolean argument.

The input data to signal_from_telomeres_average() is the output of signal_from_telomeres(), which contains only sub-telomeric regions. This means that the complete wiggle data, required to calculate genome-wide signal averages is not part of the input. The user should do this independently, using the complete wiggle data.


UnicodeDecodeError when executing ays blueprint

Traceback (most recent call last):
File "/opt/jumpscale8/bin/ays", line 339, in
cli()
File "/usr/local/lib/python3.5/dist-packages/click/core.py", line 716, in call
return self.main(*args, **kwargs)
File "/usr/local/lib/python3.5/dist-packages/click/core.py", line 696, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.5/dist-packages/click/core.py", line 1057, in invoke
Command.invoke(self, ctx)
File "/usr/local/lib/python3.5/dist-packages/click/core.py", line 889, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python3.5/dist-packages/click/core.py", line 534, in invoke
return callback(*args, **kwargs)
File "/opt/jumpscale8/bin/ays", line 22, in cli
atyourservice = j.atyourservice.get(j.sal.fs.getBaseName(cwd), cwd)
File "/opt/jumpscale8//lib/JumpScale/baselib/atyourservice/AtYourServiceFactory.py", line 69, in get
if name not in self.repos:
File "/opt/jumpscale8//lib/JumpScale/baselib/atyourservice/AtYourServiceFactory.py", line 86, in repos
self._repos[name] = AtYourServiceRepo(name, path)
File "/opt/jumpscale8/lib/JumpScale/baselib/atyourservice/AtYourServiceRepo.py", line 45, in init
self._load_blueprints()
File "/opt/jumpscale8/lib/JumpScale/baselib/atyourservice/AtYourServiceRepo.py", line 225, in _load_blueprints
self._blueprints[path] = Blueprint(self, path=path)
File "/opt/jumpscale8/lib/JumpScale/baselib/atyourservice/Blueprint.py", line 23, in init
self.content = j.sal.fs.fileGetContents(path)
File "/opt/jumpscale8//lib/JumpScale/sal/fs/SystemFS.py", line 1317, in fileGetContents
data = fp.read()
File "/usr/lib/python3.5/encodings/ascii.py", line 26, in decode
return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0x8d in position 16: ordinal not in range(128)

What was in the blueprint?

Not sure what the exact blueprint was, I only know that it was one for deploying a VDC.
I remember that you then solved a problem by removing a swap file of the blueprint, maybe that was the cause. Feel free to close @rkhamis


1.4.6 machines stop working in smp

Issue description:
machines randomly stop working, for example the solderer
What happens:
machines randomly stop working, for example the solderer
What you expected to happen:
expected the solderer to keep working
Steps to reproduce:

attach more than one solderer
I used hoppers on top and below to automate them
then without rhyme or reason they stop working
tested it in ssp, no issues
...

Version (make sure you are on the latest version before reporting):

Minecraft: 1.11.2
Forge: 2296
Refined Storage: 1.4.6

Does this issue occur on a server? yes
If a (crash)log is relevant for this issue, link it here:
There are no crashes or server console spam do you have a switch to make this verbose so I can help debug this better?

Bad reproduction steps.


Issue with airplane mode on zik 3

Hi there,
I've installed this tool on my mac, and turned on the airplane mode.
Since then I can't turn off the airplane mode, the bluetooth is not working anymore, and headphone is not detected by my smartphone and my mac (even when connected by cable).
Any idea about what's happening ?
Thanks

On your smartphone, try switching on and off the airplane mode, hopefully, that should fix it; Also, try turning the headphones off and on (or take out the batteries)


Fails when Device is offline when first added.

If a device reports itself offline when first added, and thus the latitude and longitude are blank, then FindFriendsMini fails with an error:
   FindFriendsMini Error           Error Within checkHomeOther:
Traceback (most recent call last):
  File "plugin.py", line 1458, in checkHomeOther
    iDevLatitude = float(dev.states['latitude'])
ValueError: could not convert string to float:

This causes GeoFences to fail to properly be updated (because the error gets thrown while trying to update GeoFence status).

Hi
Thanks - the exception is caught (just Indigo 7 logging - enables to show in great detail)
So should continue to work and once online should be okay.
But will make a bit more tidy!
G

Agreed, once online all will be good.  The problem is that if the device is offline from the start, and stays offline for some time never going online, GeoFence updates are broken.
Thanks!

Thanks
Have updated to 0.5.8 which should fix this.
Should be there now.
G


Support limit for timeseries (native and SQL)

The timeseries query should support a limit parameter. The main motivation for this is for Druid SQL: people sometimes write a query with a limit X that really should be a timeseries, but we can't plan it as one, since timeseries doesn't support limits.

A good test case for this would be a SQL query like:
select floor(__time to day), count(*) from foo limit 2

Which should plan into a timeseries query with a limit: 2 parameter. But today, it plans into, I think, a topN.

Fixed by https://github.com/apache/incubator-druid/pull/5931


Fix negated query for mc search

In simple search negated queries were still being searched against.
Fix simple search negation query logic:

(now using OR to separate negations)
Results do not have negated terms:

closes #667

Accidentally kept working on this branch 😅
Add all source meta data to show page, add '?' if no metadata found

Fix bug to be able to delete a url_search_string from a source.


Request: Please implement the payeeEmail for Android & iOS

The payeeEmail attribute is an optional property of the class PayPalPayment. Can you please implement it, it is needed in our project to send the payment to another email address.
Regards

Please attach docs for both Android / iOS


gvisor: init at 2018-11-10

Motivation for this change
Add a package for the gvisor container runtime sandbox.  This was requested in #39889, but there were some problems with Bazel at the time.  I've managed to get this working, but I'd appreciate feedback on how I've done so.  In short: there's two derivations here; one that is a fixed-output derivation produced by running bazel sync to download all dependencies and making them deterministic, and a second that uses the above derivation along with the source in order to build the actual output binary.  At the end of the whole process, gvisor is runnable:
$ /nix/store/iag5vgl51alqmirabvz5ij9yfp6kwmby-gvisor-2018-11-10/bin/runsc --help
Usage: runsc <flags> <subcommand> <subcommand args>

Subcommands:
	checkpoint       checkpoint current state of container (experimental)
	create           create a secure container
	delete           delete resources held by a container
	events           display container events such as OOM notifications, cpu, memory, and IO usage statistics

I haven't yet tested this with Docker, so I'll try to do that shortly.
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[x] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[x] Determined the impact on package closure size (by running nix path-info -S before and after)
[x] Fits CONTRIBUTING.md.

cc @dtzWill and @q3k (on the original issue)
cc @mboes (Bazel maintainer - feedback appreciated!)

I tried testing this with Docker, but it looks like the CONFIG_CGROUP_PERF kernel option isn't enabled in the default Nixpkgs kernel, leading to some form of incompatibility between gvisor and Docker.  I initially get the following error:
error creating container: error configuring cgroup: mkdir /sys/fs/cgroup/perf_event: read-only file system

I can remount the cgroup filesystem as rw (sudo mount -o remount,rw /sys/fs/cgroup), but when doing that or patching gvisor to remove that cgroup from the controller set, I get the error:
unable to find "perf_event" in controller set: unknown.

I don't have time to rebuild my kernel with that cgroup option right now, but I'll try to get it to it soon, unless someone else wants to have a try!

@orivej - After applying #50225, and setting virtualisation.docker.extraOptions = "--add-runtime=runsc=/nix/store/[...]-gvisor-2018-11-10/bin/runsc";, I'm able to run docker run --runtime=runsc -it ubuntu /bin/bash, apt-get install things from within the container, and generally do things. I also confirmed via ps that gvisor was running! :tada:

The goal is to not have to download dependencies with Bazel (bazel sync) at application build time.

I tried a couple ways to do this, but wasn't successful.  You can use native.existing_rules() to iterate over all repository rules in a Bazel workspace, but turning those into things that Nix can fetch is pretty tricky. Especially since you can't just search for a standard set of rules that access the network, since e.g. rules_go has some repository rules that run custom commands to fetch dependencies. I suspect that bazel sync is probably the best we're going to get, honestly.

Do you know if there is a more simple way to prefetch dependencies for Bazel builds in nixpkgs?

bazel sync is able to produce some kind of lock file, but it’s kinda verbose and not in a well-known format, but skylark. It might be possible to eval it with a python interpreter and spew out some json.

Current state: I've submitted a patch to bazel-gazelle to make the helper build tools deterministic (https://github.com/bazelbuild/bazel-gazelle/pull/382) which has been merged, but that's not sufficient; I'm currently chasing down some Nix paths in the dependency output. Most of them are local configuration from the environment, and we can just remove them (rm -rf $out/local_config*), but there's one particular problem that I'm running into:
Our Go compiler has patches[1][2] that replace the absolute /etc/services, /etc/protocols, and /usr/share/zoneinfo paths with Nix store paths. This, however, means that we cannot use a Go binary in a fixed-output derivation, since the binary will contain paths from the Nix store and thus the fixed-output hash will change if those paths ever do. Anyone have any idea what we normally do in cases like this?  Or should we just assume that this particular problem is a lost cause, and find some other way of building these binaries?
(also, holy hell is this rapidly turning into something more complicated than I'd originally expected :stuck_out_tongue:)

This was super annoying, but: I've just force-pushed an update that successfully builds gvisor by manually fetching all dependencies with Nix. It's especially annoying since rules_go applies patches to some third-party libraries, so we have to manually apply those ourselves too, or the build will fail. However: this builds properly for me, now.
@Profpatsch and @nlewo - thoughts on this new approach?

Just pushed an alternate version; this is now generated by a very WIP script that will parse the Bazel resolved-dependencies file and attempt to convert it to a Nix file. It's pretty hacky, but I'm heading to bed and figured I'd drop it here for now!

thoughts on this new approach?

I really like it. Would be nice to split out the parser to get a generic transformation from bazel lockfile to nixpkgs package. Of course then the generated code must be overridable, I can help with that if you want. See https://github.com/Profpatsch/yarn2nix/tree/master/nix-lib for an example on how that can be done (there might be some code-reuse possible).

@andrew-d Wondered if you ever continued with this? I really like the idea of running services on my NixOS machine within gVisor KVM containers.

@benpye - The short answer is "not yet"; I'm currently trying my hand at a slightly more generic "Bazel to Nix" translator, and once I get that working will update this PR with the generated code. This branch does work, though, if you want to apply it to a local fork!

@andrew-d anything you need help with? This is rad.


(doc) Update the contribution guide

This updates the contribution doc according to the changes on #10476. Next week I'll take a look to consolidate the contribution documentation.


Associating an API model to a Lambda function should enable a request body validator.

Description:
With the recent addition of Models to the AWS::Serverless::Api resource I began working on removing all of my Swagger definitions from an API and rely only on the SAM resources. I rely on request body validation with API Gateway to verify good JSON payloads are bing provided before my function code is invoked. To me, building a serverless API, this is the key feature, behavior, and reason for associating a model with a method-path.
When I went to test an API configured using only SAM resources I found that this is not the behavior when the API Gateway is created. The request body validator is not enabled even though I am supplying a model/schema. From my perspective I don't see how adding models provides functional value to my applications if they aren't being used to validate requests - it's functionality I have to move back into my codebase now, or I have to go back to defining the entire Swagger doc with all of the API Gateway extensions again.
Steps to reproduce the issue:
Create a AWS::Serverless::Api resource and associate a model with it.
Create a AWS::Serverless::Function resource and associate it with a method-path and the model.
Observed result:
The model is associated with that method-path, but request body validation is not enabled. The underlying Lambda functions are invoked even if invalid requests are being performed.
Expected result:
The model is associated with the method-path and request body validation is enabled. The underlying Lambda functions are not invoked when invalid requests are performed.

@brysontyrrell This is a known api gateway gap that is being tracked in https://github.com/awslabs/serverless-application-model/issues/931#issuecomment-534463087. Closing this issue in favor of #931


list_node and show_node rest api

The call for list_node and show_node have the same url /node/<is_free> and /node/<node> respectively.  Easy change just wanted to document.  I will change list_node to /node/list/<is_free>.

@matsuuran I am wondering is this addressed by #619 ? I am a little confused about the first sentence.

@shwsun
lets just say you have a node named "all" or "free"  you would have a collision on the rest call.  It is just a small edge case

I see. It will cause a collision and we have to solve it. However, I would more like to keep the list_nodes rest call just like other list_xxx calls. Any other ideas?
//cc @zenhack @henn @SahilTikale @gsilvis

Could make the all/free a query parameter, e.g. to list free nodes:
GET /node?state=free

Or something. This would also make it easy to add more filters in a backwards-compatible way, and if we wanted to we could coalesce things like list_project_nodes:
GET /node?project=foo

But whether we want to do that is it's own discussion.

Actually this problem is caused by mistake, we should have GET /nodes for list calls, and GET /node/<nodename> for show calls. #635 solves this and it is no longer a concern for us.


MiniTransform Bug?

I ran into an issue when uploading my own minimesh, but it seems to be a quick fix (if the fix is correct) so I figured I would write it down here.
In the MiniTransform code:
https://github.com/twak/chordatlas/blob/b77561b4460c92c1fee3f74ae913d76ffc9e0817/src/org/twak/readTrace/MiniTransform.java#L120
Should this line actually be:
double edgeLength = Math.pow( volume / ( (double)count / targetCount ), 0.3333 );
which casts count to a double before dividing? Otherwise if count < targetCount, we are dividing by 0 and edgeLength becomes infinity.

I agree this is a bug!
This might be a bit smoother if I have your dataset? (Obviously) not many people have used this pipeline except me...

fixed here: https://github.com/twak/chordatlas/commit/df161340709622352feb98e3eff18c2732f03f52


How can i use fastcgi_pass

Hi,
i would like to use npm as Proxy for my Nextcloud FPM.
But, i can i configure fastcgi_pass in the ProxyManager?
Has someone running Nextcloud:FPM with Proxymanager?
Regards
Hermann

Unfortunately npm is not intended to be used that way currently. But the proxy_pass used when you create a normal proxy host should work just fine.

I think the fastcgi_Pass option has to become law here. Or does proxy_pass also work with FPM?

Is there a documentation how i can use advanced configuration?
Is it possible to add this Parameter to the conf File manualy?

Is it? Isn't fastcgi_pass meant if you want to forward within the same host? I'm pretty sure proxy_pass should also work, just go ahead and try it.
Using the advanced config you can't remove things which are generated by npm, and the proxy_pass directive is always automatically included. You could manually edit the .conf file within the container manually found at /data/nginx/proxy_host/x.conf (x being the id of the proxy host, found when clicking on the three dots next to the proxy host inside the admin ui). There you could replace proxy_pass with fastcgi_pass and restart the container. But as soon as you edit the proxy host and save again, the config will be overwritten.

Hmm ... proxy_pass doesn't work for me. Or I'm doing something wrong.
I have created a new proxy host with the IP from the NC FPM IMage. Unfortunately no access to it.

Unfortunately then the only way is to manually edit the .conf files inside the container. You would have to wait for feature request https://github.com/jc21/nginx-proxy-manager/issues/65 otherwise, but I don't think it is going to happen in v2.

Is there any way we could do this?
Sorry for commenting on this old issue, but I really need to use "fastcgi_pass" for my Wordpress site using php-fpm.
Normal Wordpress setup without php-fpm and with just PHP works fine. But when I use Wordpress php-fpm version, I get 502 error when visiting my domain.
I would really appreciate if someone could tell me a way to use "fastcgi_pass" instead of proxy_pass

Hello, just joining this thread. I would like to see this feature too. Advantage is that i don't need to download containers with apache version of solution but with fastcgi. Why to have many servers when it can be solved with one ;)

Yeah... why a app made for "making nginx easy" doesn't... make thing easy for nginx ?
I really don't understand. Why it's SO HARD to just.. add some config to nginx config on domains ? Are we not free ?


{Linked:Bug9781456} [Adaptive Cards> Designer]: Screen reader is not announcing any information after expanding or collapsing the 'Oct 22' buttons.

Target Platforms
Other
SDK Version
1.5
Application Name
Adaptive Cards
Problem Description
Test Environment:
OS Version: 22H2 (Build 25324.1011)
Browser: Edge Dev (113.0.1774.3)
Screen Reader: NVDA
Repro Steps:

Open the above URL in edge dev browser.
Navigate to the 'Designer' tab through tab key and activate it.
Navigate to the 'Sample data Editor' and ' Card payload editor' and attach the provided JSON file.
Navigate to 'Preview mode' and activate it.
Navigate to the 'Oct 22' button and press enter key and observe the issue.

Actual Result:
While navigating to the 'Oct 22' button, screen reader is not conveying the state as 'Collapsed or expanded'. Also, after activating it on renavigating on the same control screen reader is not conveying the state.
Expected Result:
Screen Reader should convey the state as 'Collapsed or Expanded' for 'Oct 22' button.
User Impact:
Screen reader users may affect if it does not give accurate announcement to the action on the screen.
WCAG Reference:
https://www.w3.org/WAI/WCAG21/Understanding/info-and-relationships.html
"Have feedback to share on Bugs? Please help fill Trusted Tester Bug Feedback (office.com) “.
Screenshots
https://user-images.githubusercontent.com/95913748/230363346-05cb11c1-3e6e-49d1-a031-da32dc56548d.mp4
stock-awards-vesting-screen json.zip
Card JSON
Sample Code Language
No response
Sample Code
No response

Hi @vagpt,
This issue can be resolved by switching from ToggleVisibility to ShowCard. The ShowCard action has the accessibility behavior you are looking for and seems to be good for this use case.
Here is a link to the documentation: https://adaptivecards.io/explorer/Action.ShowCard.html
Also in the future, ToggleVisibility is a deprecated feature, and we no longer support it.
Let me know if you have any questions or would like an updated payload. Thanks!


ERROR  Nitro is not initialized yet. You can call useNitro() only after ready hook.

Environment

Operating System: Darwin
Node Version:     v14.17.6
Nuxt Version:     2.16.0-27720022.54e852f
Nitro Version:    0.5.4
Package Manager<EMAIL_ADDRESS>Builder:          webpack
User Config:      bridge, serverHandlers, devServerHandlers, buildModules
Runtime Modules:  -
Build Modules<EMAIL_ADDRESS>

Reproduction
https://stackblitz.com/edit/github-nrsxrr?file=package.json,nuxt.config.ts

yarn build

Describe the bug
Build command fails with ERROR  Nitro is not initialized yet. You can call useNitro() only after ready hook. error. This bug occurred after rc.11.
I just set nitro: falsein nuxt config.
Additional context
No response
Logs
No response

I am having the same issue declaring the nitro property in the bridge config.

I am also having this issue when attempting to run with Nitro set to false.

I am trying to upgrade a nuxt app v2 using bridge and I am facing the same issue:

nuxt.config.ts:

@marabesi What version of Bridge are you using?


FS-4529 initial db

https://dluhcdigital.atlassian.net/browse/FS-4529
https://dluhcdigital.atlassian.net/browse/FS-4530
Refactoring of the FAB prototype to include DB entities. Model classes, migrations, and a usable (not designed!) UI to view this data.
To run locally, follow the instructions in the README. Recommend using the devcontainer setup, and then creating a docker compose file to start up this and the form runner.
Have added notes to the readme, but the stuff circled in red on the below screenshot is what's new in this PR and uses the database. Everything above that is the old drag and drop prototype, which has not been fully updated to use the DB so is not completely working at present. This will be fixed in later PRs as more functionality is updated to use the DB.

lovely work @srh-sloan as always.
Just left a few comments and possibly talking point.
Rather that comment on page there are just a few other UI things that would be nice:

Hint on data inputs
Specific field inputs (datetime)
-Implement back button on journey
CSS issue on button placement
Static file loading (as discussed this morning on slack)
A way of deleting sections/forms etc

I realise these may be covered in other tickets that i havnt seen


add probot dco default conf

This commit is just at test for the probot DCO plugin, which enforces signoff for all prs or fails and provide instructions on how to fix them.  By adding this conf, members of the organization can skip the signoff (or should be able to, we will test). And of course, we have to discuss adding -signoff as a good default practice for whole org.

This commit is just at test for the probot DCO plugin, which enforces signoff for all prs or fails and provide instructions on how to fix them.  By adding this conf, members of the organization can skip the signoff (or should be able to, we will test). And of course, we have to discuss adding -signoff as a good default practice for whole org.


xAxis on multiChart

I can't seem to be able to configure my xAxis on a multiChart.
I try to specify
ticks: 9

or
xDomain: [0, 9]

in several different placer (chart, chart.xAxis, chart.lines, chart.lines1, chart.lines1.xAxis...), but none of them work.
Even by using the live multiChart exemple on the website, I can't seem to be able to manage it.
Most of the time, I end up with no xAxis ticks at all! I have no idea what's going on anymore.
Anybody got a working example of such a thing?
Are there restrictions on the multiChart type when setting scales?
If there are, I suppose I shouldn't use multiChart, but, in that case, is there a way to have two yAxis on a cumulativeLineChart (that's why I'm using the multiChart)?

this is example of using ticks. note, the number of ticks may be differ from specified value, since d3 always recalculates this value.
also, cumulativeLineChart doesn't support the second yAxis.

How to set x axis range in multiChart?


[CodeCompletion] Context type analysis for default argument initializer

This enables implicit member expression completion at default argument initializer position. e.g.
func foo(val: MyEnum = .<HERE>) {

rdar://problem/51037538

@swift-ci Please smoke test

It seems this doesn't work at default argument for enum element. e.g.
enum MyEnum {
  case something(thing: Something = .<HERE>)
}

I will rethink.

@swift-ci Please smoke test

@swift-ci Please smoke test


More aws tags

Add some more meta-data tags to agents running on aws.
This switches ec2 meta-data to a single instance identity document which should be slightly more efficient, and adds some more useful meta-data including region and az when running on ec2.
This also upgrades the ecs metadata library we were using so it can grab task meta-data as well as container meta-data, and uses that to add availability zone, launch type (fargate or not) and cpu/memory limits ("size") there as well.
AWS have some example data here:
https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task-metadata-endpoint-v3.html#task-metadata-endpoint-v3-examples
https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task-metadata-endpoint-v4.html#task-metadata-endpoint-v4-examples
and the ecs meta-data module is a useful reference:
https://github.com/BrunoScheufler/aws-ecs-metadata-go/blob/master/v3.go
https://github.com/BrunoScheufler/aws-ecs-metadata-go/blob/master/v4.go
The best way to test this is probably to run an agent in these environments. Does anyone have a handy way to do that?
Fixes PDP-627.

I built an agent from this branch and tried it on an EC2 instance:
$ ./buildkite-agent start --token=xxx --tags-from-ec2-meta-data=true --build-path=~/bk/

   _           _ _     _ _    _ _                                _
  | |         (_) |   | | |  (_) |                              | |
  | |__  _   _ _| | __| | | ___| |_ ___    __ _  __ _  ___ _ __ | |_
  | '_ \| | | | | |/ _` | |/ / | __/ _ \  / _` |/ _` |/ _ \ '_ \| __|
  | |_) | |_| | | | (_| |   <| | ||  __/ | (_| | (_| |  __/ | | | |_
  |_.__/ \__,_|_|_|\__,_|_|\_\_|\__\___|  \__,_|\__, |\___|_| |_|\__|
                                                 __/ |
 https://buildkite.com/agent                    |___/

2023-03-06 11:06:12 NOTICE Starting buildkite-agent v3.44.0 with PID: 744
2023-03-06 11:06:12 NOTICE The agent source code can be found here: https://github.com/buildkite/agent
2023-03-06 11:06:12 NOTICE For questions and support, email us at<EMAIL_ADDRESS>2023-03-06 11:06:12 INFO   Fetching EC2 meta-data...
2023-03-06 11:06:12 INFO   Successfully fetched EC2 meta-data
2023-03-06 11:06:12 INFO   Registering agent with Buildkite...
2023-03-06 11:06:13 INFO   Successfully registered agent "ip-172-31-38-184" with tags [queue=default, aws:instance-id=i-0bb667bfcef706257, aws:instance-type=t3.large, aws:ami-id=ami-05cc01f6bf715d472, aws:availability-zone=ap-southeast-4c, aws:region=ap-southeast-4, aws:instance-life-cycle=on-demand]
2023-03-06 11:06:13 INFO   Starting 1 Agent(s)
2023-03-06 11:06:13 INFO   You can press Ctrl-C to stop the agents
2023-03-06 11:06:13 INFO   ip-172-31-38-184 Connecting to Buildkite...
2023-03-06 11:06:14 INFO   ip-172-31-38-184 Waiting for work...

It looks like this in the UI:

I tried an ECS/Fargate task with the following settings.

Fargate Platform 1.4 ( I believe this uses the V4 metadata path ):
ENV vars:

BUILDKITE_AGENT_TOKEN
BUILDKITE_AGENT_TAGS_FROM_ECS_META_DATA

I It registers like this:
timestamp,message
1678103381902,   _           _ _     _ _    _ _                                _
1678103381902,  | |         (_) |   | | |  (_) |                              | |
1678103381902,  | |__  _   _ _| | __| | | ___| |_ ___    __ _  __ _  ___ _ __ | |_
1678103381902,  | '_ \| | | | | |/ _` | |/ / | __/ _ \  / _` |/ _` |/ _ \ '_ \| __|
1678103381902,  | |_) | |_| | | | (_| |   <| | ||  __/ | (_| | (_| |  __/ | | | |_
1678103381902,"  |_.__/ \__,_|_|_|\__,_|_|\_\_|\__\___|  \__,_|\__, |\___|_| |_|\__|"
1678103381902,                                                 __/ |
1678103381902, https://buildkite.com/agent                    |___/
1678103381902,
1678103381902,2023-03-06 11:49:41 NOTICE Starting buildkite-agent v3.44.0 with PID: 7
1678103381902,2023-03-06 11:49:41 NOTICE The agent source code can be found here: https://github.com/buildkite/agent
1678103381902,"2023-03-06 11:49:41 NOTICE For questions and support, email us at<EMAIL_ADDRESS>1678103381902,2023-03-06 11:49:41 INFO   Configuration loaded path=/buildkite/buildkite-agent.cfg
1678103381902,2023-03-06 11:49:41 INFO   Fetching ECS meta-data...
1678103381906,2023-03-06 11:49:41 INFO   Successfully fetched ECS meta-data
1678103381906,2023-03-06 11:49:41 INFO   Registering agent with Buildkite...
1678103382965,"2023-03-06 11:49:42 INFO   Successfully registered agent ""ip-172-31-9-22.ap-southeast-4.compute.internal-1"" with tags [queue=default, ecs:task-arn=arn:aws:ecs:ap-southeast-4:111111111111:task/jhealy-test/139624172e3540e6af4a5da51e1ac3d2, ecs:cpu-limit=1, ecs:memory-limit=3072, ecs:container-name=bkagent, ecs:image=111111111111.dkr.ecr.ap-southeast-4.amazonaws.com/bkagent:1, ecs:availability-zone=ap-southeast-4b, ecs:launch-type=FARGATE]"
1678103382965,2023-03-06 11:49:42 INFO   Starting 1 Agent(s)
1678103382965,2023-03-06 11:49:42 INFO   You can press Ctrl-C to stop the agents
1678103382965,2023-03-06 11:49:42 INFO   ip-172-31-9-22.ap-southeast-4.compute.internal-1 Connecting to Buildkite...
1678103383839,2023-03-06 11:49:43 INFO   ip-172-31-9-22.ap-southeast-4.compute.internal-1 Waiting for work...

I also started an ECS/Fargate task using platform 1.3 and 0.5 CPU. This was the only surprising result - it registered like this (note ecs:availability-zone=true):
1678103616955,  | |         (_) |   | | |  (_) |                              | |
1678103616955,  | |__  _   _ _| | __| | | ___| |_ ___    __ _  __ _  ___ _ __ | |_
1678103616955,  | '_ \| | | | | |/ _` | |/ / | __/ _ \  / _` |/ _` |/ _ \ '_ \| __|
1678103616955,  | |_) | |_| | | | (_| |   <| | ||  __/ | (_| | (_| |  __/ | | | |_
1678103616955,"  |_.__/ \__,_|_|_|\__,_|_|\_\_|\__\___|  \__,_|\__, |\___|_| |_|\__|"
1678103616955,                                                 __/ |
1678103616955, https://buildkite.com/agent                    |___/
1678103616955,
1678103616956,2023-03-06 11:53:36 NOTICE Starting buildkite-agent v3.44.0 with PID: 7
1678103616956,2023-03-06 11:53:36 NOTICE The agent source code can be found here: https://github.com/buildkite/agent
1678103616956,"2023-03-06 11:53:36 NOTICE For questions and support, email us at<EMAIL_ADDRESS>1678103616956,2023-03-06 11:53:36 INFO   Configuration loaded path=/buildkite/buildkite-agent.cfg
1678103616956,2023-03-06 11:53:36 INFO   Fetching ECS meta-data...
1678103616960,2023-03-06 11:53:36 INFO   Successfully fetched ECS meta-data
1678103616960,2023-03-06 11:53:36 INFO   Registering agent with Buildkite...
1678103618004,"2023-03-06 11:53:38 INFO   Successfully registered agent ""ip-172-31-19-234.ap-southeast-4.compute.internal-1"" with tags [queue=default, ecs:launch-type=FARGATE, ecs:task-arn=arn:aws:ecs:ap-southeast-4:111111111111:task/jhealy-test/a8d52f1ce90a4484a336bc0b41040080, ecs:cpu-limit=0.5, ecs:memory-limit=3072, ecs:container-name=ecs-bkagent-4-bkagent-a4ddf9b582faf5bf1300, ecs:image=111111111111.dkr.ecr.ap-southeast-4.amazonaws.com/bkagent:1, ecs:availability-zone=true]"
1678103618004,2023-03-06 11:53:38 INFO   Starting 1 Agent(s)
1678103618004,2023-03-06 11:53:38 INFO   You can press Ctrl-C to stop the agents
1678103618004,2023-03-06 11:53:38 INFO   ip-172-31-19-234.ap-southeast-4.compute.internal-1 Connecting to Buildkite...
1678103618868,2023-03-06 11:53:38 INFO   ip-172-31-19-234.ap-southeast-4.compute.internal-1 Waiting for work...

I also started an ECS/EC2 task, it registered like this (note ecs:launch-type=EC2):
1678107702843,   _           _ _     _ _    _ _                                _
1678107702843,  | |         (_) |   | | |  (_) |                              | |
1678107702843,  | |__  _   _ _| | __| | | ___| |_ ___    __ _  __ _  ___ _ __ | |_
1678107702843,  | '_ \| | | | | |/ _` | |/ / | __/ _ \  / _` |/ _` |/ _ \ '_ \| __|
1678107702843,  | |_) | |_| | | | (_| |   <| | ||  __/ | (_| | (_| |  __/ | | | |_
1678107702843,"  |_.__/ \__,_|_|_|\__,_|_|\_\_|\__\___|  \__,_|\__, |\___|_| |_|\__|"
1678107702843,                                                 __/ |
1678107702843, https://buildkite.com/agent                    |___/
1678107702843,
1678107702843,2023-03-06 13:01:42 NOTICE Starting buildkite-agent v3.44.0 with PID: 7
1678107702843,2023-03-06 13:01:42 NOTICE The agent source code can be found here: https://github.com/buildkite/agent
1678107702843,"2023-03-06 13:01:42 NOTICE For questions and support, email us at<EMAIL_ADDRESS>1678107702843,2023-03-06 13:01:42 INFO   Configuration loaded path=/buildkite/buildkite-agent.cfg
1678107702843,2023-03-06 13:01:42 INFO   Fetching ECS meta-data...
1678107702849,2023-03-06 13:01:42 INFO   Successfully fetched ECS meta-data
1678107702849,2023-03-06 13:01:42 INFO   Registering agent with Buildkite...
1678107703727,"2023-03-06 13:01:43 INFO   Successfully registered agent ""9ba570440ce8-1"" with tags [queue=default, ecs:launch-type=EC2, ecs:task-arn=arn:aws:ecs:ap-southeast-4:111111111111:task/jhealy-test/0e533b9b1f3d4cd19feafe0cdaa6673f, ecs:cpu-limit=1, ecs:memory-limit=3072, ecs:container-name=ecs-bkagentec2-2-bkagent-8ab89bd88eb1aae1c701, ecs:image=111111111111.dkr.ecr.ap-southeast-4.amazonaws.com/bkagent:1, ecs:availability-zone=ap-southeast-4c]"
1678107703727,2023-03-06 13:01:43 INFO   Starting 1 Agent(s)
1678107703727,2023-03-06 13:01:43 INFO   You can press Ctrl-C to stop the agents
1678107703727,2023-03-06 13:01:43 INFO   9ba570440ce8-1 Connecting to Buildkite...
1678107704600,2023-03-06 13:01:44 INFO   9ba570440ce8-1 Waiting for work...

Interestingly, if I specify BUILDKITE_AGENT_TAGS_FROM_EC2_META_DATA and BUILDKITE_AGENT_TAGS_FROM_ECS_META_DATA on ECS/EC2 tasks the agent is able to fetch the instance and ECS metadata:
2023-03-06 13:11:16 INFO   Successfully registered agent ""50fb3f23749e-1"" with tags [queue=default, aws:instance-id=i-06a1e7de8db1129c2, aws:instance-type=t3.large, aws:ami-id=ami-0cd8fb3ad4a0e425e, aws:availability-zone=ap-southeast-4c, aws:region=ap-southeast-4, aws:instance-life-cycle=on-demand, ecs:memory-limit=3072, ecs:container-name=ecs-bkagentec2-3-bkagent-ccbb8ea8b4aef99b3b00, ecs:image=111111111111.dkr.ecr.ap-southeast-4.amazonaws.com/bkagent:1, ecs:availability-zone=ap-southeast-4c, ecs:launch-type=EC2, ecs:task-arn=arn:aws:ecs:ap-southeast-4:111111111111:task/jhealy-test/5e9a276eb885484ca776fd7a35712590, ecs:cpu-limit=1]"

We end up with aws:region=ap-southeast-4, aws:availability-zone=ap-southeast-4c and ecs:availability-zone=ap-southeast-4c.
If I specify the same two ENV vars on ECS/Fargate, the agent is unable to fetch the instance metadata, but ultimately still registers successfully.
1678108747386,2023-03-06 13:19:07 NOTICE Starting buildkite-agent v3.44.0 with PID: 8
1678108747386,2023-03-06 13:19:07 NOTICE The agent source code can be found here: https://github.com/buildkite/agent
1678108747386,"2023-03-06 13:19:07 NOTICE For questions and support, email us at<EMAIL_ADDRESS>1678108747386,2023-03-06 13:19:07 INFO   Configuration loaded path=/buildkite/buildkite-agent.cfg
1678108747386,2023-03-06 13:19:07 INFO   Fetching EC2 meta-data...
1678108747657,2023-03-06 13:19:07 WARN   EC2MetadataRequestError: failed to get EC2 instance identity document
1678108747657,caused by: RequestError: send request failed
1678108747657,"caused by: Get ""http://<IP_ADDRESS>/latest/dynamic/instance-identity/document"": dial tcp <IP_ADDRESS>:80: connect: invalid argument (Attempt 1/5 Retrying in 2s)"
1678108749939,2023-03-06 13:19:09 WARN   EC2MetadataRequestError: failed to get EC2 instance identity document
1678108749939,caused by: RequestError: send request failed
1678108749939,"caused by: Get ""http://<IP_ADDRESS>/latest/dynamic/instance-identity/document"": dial tcp <IP_ADDRESS>:80: connect: invalid argument (Attempt 2/5 Retrying in 2s)"
1678108752188,2023-03-06 13:19:12 WARN   EC2MetadataRequestError: failed to get EC2 instance identity document
1678108752188,caused by: RequestError: send request failed
1678108752188,"caused by: Get ""http://<IP_ADDRESS>/latest/dynamic/instance-identity/document"": dial tcp <IP_ADDRESS>:80: connect: invalid argument (Attempt 3/5 Retrying in 2s)"
1678108754471,2023-03-06 13:19:14 WARN   EC2MetadataRequestError: failed to get EC2 instance identity document
1678108754471,caused by: RequestError: send request failed
1678108754471,"caused by: Get ""http://<IP_ADDRESS>/latest/dynamic/instance-identity/document"": dial tcp <IP_ADDRESS>:80: connect: invalid argument (Attempt 4/5 Retrying in 2s)"
1678108756732,2023-03-06 13:19:16 WARN   EC2MetadataRequestError: failed to get EC2 instance identity document
1678108756732,caused by: RequestError: send request failed
1678108756732,"caused by: Get ""http://<IP_ADDRESS>/latest/dynamic/instance-identity/document"": dial tcp <IP_ADDRESS>:80: connect: invalid argument (Attempt 5/5)"
1678108756732,2023-03-06 13:19:16 ERROR  Failed to fetch EC2 meta-data: EC2MetadataRequestError: failed to get EC2 instance identity document
1678108756732,caused by: RequestError: send request failed
1678108756732,"caused by: Get ""http://<IP_ADDRESS>/latest/dynamic/instance-identity/document"": dial tcp <IP_ADDRESS>:80: connect: invalid argument"
1678108756732,2023-03-06 13:19:16 INFO   Fetching ECS meta-data...
1678108756736,2023-03-06 13:19:16 INFO   Successfully fetched ECS meta-data
1678108756736,2023-03-06 13:19:16 INFO   Registering agent with Buildkite...
1678108757615,"2023-03-06 13:19:17 INFO   Successfully registered agent ""ip-172-31-2-245.ap-southeast-4.compute.internal-1"" with tags [queue=default, ecs:cpu-limit=1, ecs:memory-limit=3072, ecs:container-name=bkagent, ecs:image=111111111111.dkr.ecr.ap-southeast-4.amazonaws.com/bkagent:1, ecs:availability-zone=ap-southeast-4b, ecs:launch-type=FARGATE, ecs:task-arn=arn:aws:ecs:ap-southeast-4:111111111111:task/jhealy-test/0b4a59e201de4e08b1c5d441179cd7c8]"

That's probably all as expected, I just thought I'd record the observation.

@yob i think that makes sense - there isn't an EC2 instance to gather information from[0], so --tags-from-ec2 doesn't really make much sense.
all of this points more to the need for autotagging imo - having a good way to decide "is this tranche of tags even relevant for me" would get rid of the need to have this fail
[0] kinda sorta

@yob interesting. There are two places potentially containing availability zone for ECS. If you have easy access to spin up an ECS/EC2 task, could you please run one with something like this to dump the raw meta-data so we could take a look — even if it runs as a job on the agent it should work:
echo "Container meta-data:"
curl "$ECS_CONTAINER_METADATA_URI"
echo "Task meta-data:"
curl "$ECS_CONTAINER_METADATA_URI/task"

From ECS/Fargate docs:

Task metadata endpoint version 4 – Available for tasks that use platform version 1.4.0 or later.
Task metadata endpoint version 3 – Available for tasks that use platform version 1.1.0 or later.

My test on platform 1.4.0 has the availability zone, presumably fetched from the v4 metadata endpoint.
My test on platform 1.3.0 has "availability-zone=true", presumably because there's some issue fetching the AZ from the v3 metadata endpoint.
@sj26  I'll run some extra debugging and provide it to you

Oh, from the v3 docs

Surely not many folks are using the v1.3.0 fargate platform, its pretty old. Maybe we can skip the AZ tag in that case?

Thanks @moskyb!


[EMB-80][Improvement] Use pod styles and better style conventions

Purpose
Use ember-component-css to organize our stylesheets like the rest of the app, with a separate stylesheet for each component and route.
Summary of Changes

Stylesheets

Each component and route has a styles.scss file in their pod, which is automatically namespaced
styles/app.scss: only imports
styles/_variables.scss: variables that are available throughout the app, like colors
styles/_global.scss: global rule declarations (try not to use these ever)
styles/_mixins.scss: mixins that can be used anywhere

Conventions

BEM naming

Use ember-component-css's automatic namespace as the "block"
Our pod stylesheets typically look like: &__element,
 &__element--modifier {
 }

 &__element__subelement {
 }

 &__element__subelement--modifier {
 }

While the same classes in our templates look like{{styleNamespace}}__element
{{styleNamespace}}__element--modifier

Use variables for all colors

Stylesheet linting

~Added sass-lint, and cranked all warnings up to errors~
Added stylelint with a recommended scss ruleset
rule suggestions welcome!

Side Effects / Testing Notes
Thoughts and opinions on conventions and linting rules are more than welcome, as these decisions may haunt us for a while.
I tried not to change how anything was styled, but if someone familiar with quickfiles could go through it and let me know if I messed anything up, I'd appreciate it.
Ticket
https://openscience.atlassian.net/browse/EMB-80
Reviewer Checklist

[ ] meets requirements
[ ] easy to understand
[ ] DRY
[ ] testable and includes test(s)
[ ] changes described in CHANGELOG.md (don't have initial release yet)

Huge fan of a globally imported variables stylesheet (with smaller scope than osf-styles), and obviously of pod-styles. Really not a big fan of the BEM naming convention. For the scale of this project, specially since it will be wide but not deep (as in, lots of components, but they mostly look the same), it seems that collisions can be avoided with more judicious naming from now on, or won't actually create problems, and it would avoid lines that look like this https://github.com/CenterForOpenScience/ember-osf-web/pull/63/files#diff-d1022eb6a557bef93ce31a9dd11c6adaR4. Based on this PR alone, it seems it would make writing templates a little more complicated, and reading them a little more annoying. Not 100% against it, but it does seem like a lot of work if the only benefit is the potential class overwrite/collision down the road.

also, unrelated, big fan of the branch name here, is that following something or just a thing you do @aaxelb ?

@hmoco You make good points. I agree the extra complexity in templates is a problem, and ember-component-css already gets us the vast majority of the way toward avoiding collisions.
What about a sort of "local BEM", where we use BEM-style classes within each pod, but rely on ember-component-css to avoid broader collisions?
Something like this?

.navbar {
  &__item {
    &--active {
    }
  }
}

We'd have the information-rich class names of BEM (unlikely to collide with a 3rd-party class, easy to look at the template and know where the styles live) while avoiding the {{styleNamespace}} mess altogether.
(and the branch name is just a thing I started doing because whether it's fix/ or feature/ or whatever seems irrelevant at the moment I'm checking out a branch, while ticket number and a description are actually useful)


Player does not properly stay in the death state when spindashing into a spike underwater

Not sure if this is because of how the spike is set up, but when the player spindashes into a spike underwater, they are put into the death state for about a frame before being placed back on the ground and are able to do normal actions until the room resets.
Video of the bug in action: https://www.youtube.com/watch?v=GOzaN1x_ItU

This has been fixed as of the latest commit.


feat(tag)

Codecov Report

Merging #289 (0fe8bce) into dev (ef9231c) will decrease coverage by 0.74%.
The diff coverage is 35.00%.

@@            Coverage Diff             @@
##              dev     #289      +/-   ##
==========================================
- Coverage   92.50%   91.75%   -0.75%
==========================================
  Files         128      129       +1
  Lines        3295     3335      +40
  Branches     1096     1111      +15
==========================================
+ Hits         3048     3060      +12
- Misses        112      129      +17
- Partials      135      146      +11

Impacted Files
Coverage Δ

packages/react-magma-dom/src/theme/magma.ts
100.00% <ø> (ø)

...ackages/react-magma-dom/src/components/Tag/Tag.tsx
35.00% <35.00%> (ø)

...act-magma-dom/src/components/Dropdown/Dropdown.tsx
82.60% <0.00%> (-2.90%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ef9231c...0fe8bce. Read the comment docs.

@ccedrone I forgot one update to make.  The API Docs page needs the button in the upper-right that goes to the Design Guidelines page.

In the development of this component, pimaryInverse was needed. This ticket was resolved as part of the work: https://github.com/cengage/react-magma/issues/260