[DPE-3869] Secrets in ZK

Secrets added to Zookeeper
NOTE to reviewers*: Please see my note at https://github.com/canonical/zookeeper-operator/pull/129#issuecomment-2065505684

Please note that, despite being the author, this PR doesn not have my approval -- as it implements a known vulerabiility.
(I added the code as I was requested, even if I couldn't agree to it,)
My proposed solution is in this PR. It's essentially 5 lines of change (starting here), to remove the flaky behavior.
(All tests, typing, etc. are left intact. :green_circle: pipeline.)
Given the minor overhead vs. removing a flaky behavior, I encourage you once more to consider my proposal.
(If not, I'd be interested to know why.)
The issue comes from the inconsistent way of handling Relation Data on State objects.
If a relation is absent, we return a snapshot ({}). If the relation is present, we return a live "databag".
Even thought we provide an update method for state, it's only a redundancy. Developers can directly use the public attribute relation_data and modify contents. It will work. Most of the time.
The problematic pattern is confirmed in a new unittest, that's part of the current :green_circle: pipeline. (The complete test is here)
        # Silent falure -- there is no relation yet, thus no databag to write to
        bad_cluster_state.client.relation_data["endpoint"] = "<IP_ADDRESS>"
        # As long as there is no relation, it's always an empty dict returned
        assert bad_cluster_state.client.relation_data == {}

        # So if a function as such gets executed before the relation is joined, it has no impact

        def add_client_endpoint(client: ZKClient):
            if "endpoint" not in client.relation_data:
                client.relation_data["endpoint"] = "<IP_ADDRESS>"

        add_client_endpoint(bad_cluster_state.client)
        assert bad_cluster_state.client.relation_data == {}

        harness.add_relation(REL_NAME, "application")

        # Now we are dealing with relation data
        assert bad_cluster_state.client.relation_data == {}

        # Inconsistent behavior: now it works!
        bad_cluster_state.client.relation_data["endpoint"] = "<IP_ADDRESS>"
        assert bad_cluster_state.client.relation_data["endpoint"] == "<IP_ADDRESS>"

The nasty part is that the developer may --potentially always-- get results as expected in a deterministic test environment.
Thus assignments that never worked (due to "this-should-never-happen" real life workflows) may be discovered the hard way.
And while this may be avoidable on Zookeeper ("due to a technical coinsidence") -- Kafka has actual ClusterState properties directly corresponding to the example above. (Knowing that we unify the concept across the two charms, I can even less give my agreement to this.)
@deusebio @marcoppenheimer @zmraul @Batalex
In case this is still your preferred solution, please add a +1 on this comment for the record.
I'm waiting until the list is complete with all approvers.

@deusebio @marcoppenheimer @zmraul @Batalex
In case this is still your preferred solution, please add a +1 on this comment for the record.

+1
I had a look at the code and I don't have strong opinions on one or another implementation. The edge case in the behavior are not too critical for providing a strong nudge and I'm happy for Kafka team to make a call on this. Also, I don't feel that changing this later would be too hard either way, should we decide to be more prescriptive. We are still early in the process of getting consistent code across, and I feel that having charms with slightly different pattern can be beneficial to better see and judge pros and cons. As far as I can see, both options can be compatible and only need a small refactoring (based on the diff of the branch posted) down the road.

@deusebio @marcoppenheimer @zmraul @Batalex In case this is still your preferred solution, please add a +1 on this comment for the record. I'm waiting until the list is complete with all approvers.

+1 from me as well.
Thanks for the explanation! I acknowledge that this is an inconsistency on what we expect from relation_data. I don't mind keeping the PR as is and opening a bug with the copy/paste of the comment. We have a strong control of relations existing or not mainly through healthy checks, so I don't think it's a huge issue.

+1, but would request we remove the test_relation_data_unreliable test before approval.

+1 from me as well, and I appreciate your thorough explanation

I see 2 requests from the team on adding the fix. Should I take that as a team consensus?

Note my +1 on merging the fix, BTW ;-) (Adding my approval to my own PR :-) )

Let's focus on merging this as is if the tests pass.

@marcoppenheimer I assume you are aware of CI issues we've been struggling with throught the whole day, on a team level?
Also please note that this PR has :green_circle: CIs back to 2-3 months...
While current failures are outside or my competence.
(I'm a bit sensitive on the topic given the parituclary high attention I'm giving to pipelines... Today I re-run them about a million times on both PRs, hoping the global issue may have got fixed.)

The tests pass (or not) equally with or without the fix (pls see fix pipelines)
This PR will require a DB-libs update (+1 merge, +1 run of the tests)
The PR needs the tests (highlighting on the code weakness) to be removed (on @marcoppenheimer 's request)
Merging at this point is independent from having the fix or not. As so are the pipelines.
...didin't we plan to sync a number of PRs (K8s, Kafka VM/K8s) all at once?

Neverthless I understand the +3 (out of 5) is not a consensus -- and withrdwing my approval is preferred.
Getting back here on Monday or Tuesday with the final commit and merge (after DP-libs got merged).

BTW on this note: Tests became less table than before (i.e. past 2-3 months that I've been working on this project) 1-2 days ago.
See pipeline: https://github.com/canonical/zookeeper-operator/actions/runs/8741100693
I never had to restart the TLS pipeline 3 times before.
Now, note that:
MOST IMPORTANTLY -- NO CHANGES ON THIS CODE were applied at this point. I added requested changes in the evening of the same day. So the code (rolling back to before "Attempt to fix HA tests") is clearly not a suspect.
Now, in case the issue is the same with additional commits -- we have reasons to assume that commits may be less likely to be related to the issue.
Changes I did on the PR around the time were

rebase to main a few day before this (there were changes)
adding latest candidate version of the libs

Regarding (2) I thought that I did that 2 days ago, so this (smooth pipeline (*) is likely to have the libs. But once CI works it's easy to check.
Regaridng (1) -- is this the only pipeline where TLS tests had ot be restarted in the past 3 days?
Note: except known broken replication tests at the time -- fix is added in this PR BTW)

@marcoppenheimer I need to apologize, as I guess I've mis-understood this message, thus provided a wrong response.

Let's focus on merging this as is if the tests pass.

Now I realize that you probably proposed to merge the PR despite --unrelated-- CI failures.
I interpreted the phrasing in a wrong way (thinking that the message was pointing out pipeline failures, and if they may relate to the code). This wrong interpretation resulted in a pretty self-defensive message, which I apologize for.
Also I'm glad to see that you provided a fix for the TLS CI issue.

[SPAM] Feel free to ignore -- it's not related to the PR but to the conversation above.
I'm very 'self-conscious' and I fear I may have made wrong impressions (suggesting "professional vanity" or "territorial" attitude).
Since it's important to me not to make wrong appearances, I'd like to give more context.
I'm keen on code quality. If I see a vulnerability in my contribution, I give it a high attention, and have a difficulty to agree not to have it addressed.
I discovered this one, ~2 months ago. I proposed a different solutions to fix.
That one (my preferred solution) went along different lines (*).
Though at the time it seemed convenient for the team, later, after a deeper investigation this solution was rejected.
Instead the team had a strong requirement on contradicting aspects (***).
Therefore I crafted another fix --the one proposed above-- to fulfill the team's preferences.
What I mean is that: I had no "personal attachment" to this fix.
My intent was to offer an easy way to highlight, demonstrate and eliminate the issue -- in a way that it pleases the team.
My motivation was not to "push my code" :-) Neither would I have difficulties to accept if a proposal of mine isn't taken onboard.
My driving force was so that us, as the Data Platform Team, don't have vulnerable code.
From my perspective, any solution passing the corresponding tests could have been perfect, and have my approval.
Sorry again for the spam. My professional appearances are important for me, and I realized that my actions may transfer a wrong message without context or clarification.

(*) State class with a mandatory relation attribute (**)
(**) Just to avoid opening up this box :-) I understood that you Guys didn't want to raise Exception, and of course that part
should have been further refined (log msg, whatever), in case the rest of the proposal was otherwise desired :-)
(***) State classes with optional relation attribute that can beNone -- on which case state.relation_data should return {}

Updates: We received @marcoppenheimer 's request on the data_interfaces PR. I assume it would be evaluated by Monday afternoon (note timezones). If changes are applied, re-approved PR should be merged by Tuesday morning. After the libs are released, I can add them to this PR and merge.

Full code of the test confirming flaky behavior (removed on @marcoppenheimer 's request) is copied here for the reference:
def test_relation_data_unreliable(harness):
    """Certain patterns should be strictly avoided as long as 'relation_data' is directly exposed."""

    class BadClusterState(ClusterState):
        def __init__(self, charm, substrate):
            super().__init__(charm, substrate)

        @property
        def client_relation(self) -> Relation | None:
            return self.model.get_relation(REL_NAME)

        @property
        def client(self) -> ZKClient:
            return ZKClient(
                relation=self.client_relation,
                data_interface=self.client_provider_interface,
                component=self.client_relation.app if self.client_relation else self.model.app,
                substrate=self.substrate,
            )

    def add_client_endpoint(client: ZKClient):
        if "endpoint" not in client.relation_data:
            client.relation_data["endpoint"] = "<IP_ADDRESS>"

    with harness.hooks_disabled():
        harness.set_leader(True)
        bad_cluster_state = BadClusterState(harness.charm, "vm")

        # Silent failure -- there is no relation yet, thus no databag to write to
        bad_cluster_state.client.relation_data["endpoint"] = "<IP_ADDRESS>"
        # As long as there is no relation, it's always an empty dict returned
        assert bad_cluster_state.client.relation_data == {}

        # So if a function as such gets executed before the relation is joined, it has no impact
        add_client_endpoint(bad_cluster_state.client)
        assert bad_cluster_state.client.relation_data == {}

        harness.add_relation(REL_NAME, "application")

        # Now we are dealing with relation data
        assert bad_cluster_state.client.relation_data == {}

        # Inconsistent behavior: now assignments work
        bad_cluster_state.client.relation_data["endpoint"] = "<IP_ADDRESS>"
        assert bad_cluster_state.client.relation_data["endpoint"] == "<IP_ADDRESS>"

Merged on Team Request. Please note that my approval is missing from this PR


Sparse Mask question

Hi, I have a question about the sparsity of the weight:
After merge lora into sparse weight will change sparse weight into dense?

Hi @coldplayers, LLM-pruner is a structural method. After pruning, we get a dense model.

@VainF Thanks.


Update installed event processor version for net6 to net8 update

This was the PR that updated the tools and caused new versions to be processed. This just updates the github event processor yml files to use the new published version.

The following pipelines have been queued for testing:java - templatejava - template - testsjs - templatenet - templatenet - template - testspython - templatepython - template - tests You can sign off on the approval gate to test the release stage of each pipeline. See eng/common workflow


Lots of work on the handle bars adding styling

Nice header, looks good. Go ahead and merge, I say!

Nice header, looks good. Go ahead and merge, I say!

lol had to doublecheck which repo I was working in! Your feedback is always appreciated Daniel :P

@StaticCloud I'm good with merging this if it doesn't conflict with any of the frontend login stuff you worked on.

@StaticCloud Look good?


Adding option for specifying the a max-width for the cropper

The max width of the crop box is the width of the container. And you can define any size of max width with the cropmove event.


Please add Institut Montana Zugerberg

School website: https://www.montana-zug.ch/en
Document proving my school offers IT courses (sadly, the document is only available in German): https://montanaschule-my.sharepoint.com/personal/montana_admin_montana-zug_ch/_layouts/15/onedrive.aspx?ga=1&id=%2Fpersonal%2Fmontana%5Fadmin%5Fmontana%2Dzug%5Fch%2FDocuments%2F3%5FSG%5FSchweizersGymnasium%5FSwissSeniorHighSchool%2F21%2D22%5FLehrpläne%2F2021%5F22%5FLehrplan%5FInformatik%2Epdf&parent=%2Fpersonal%2Fmontana%5Fadmin%5Fmontana%2Dzug%5Fch%2FDocuments%2F3%5FSG%5FSchweizersGymnasium%5FSwissSeniorHighSchool%2F21%2D22%5FLehrpläne
a URL of a page or some other proof (.pdf or a screenshot are OK) showing that the university recognizes the domain which you are submitting as an official email domain for the enrolled students: Do not have such a document at my disposal. However, as you can see, the school itself is registered under the same domain as the email provided

@Joshua-H05 First stage of verification passed, official domain is montana-zug.ch as far as I verified. This is just an informational note. Pull request is still in the review. I appreciate your patience while waiting for the final decision.

Thanks a lot!


clingo.control and clingo.symbol missing in pipeline.py

Hi azreasoners,
I'm interested in your research work. However, when I run the code, I get the following error:
"File main.py:4
from pipeline import Pipeline
File pipeline.py:6
from clingo.control import Control
ModuleNotFoundError: No module named 'clingo'"
Please upload the missing files so that we can run your code.
Best Regards
Navdeep

Hi azreasoners,
I'm interested in your research work. However, when I run the code, I get the following error:
"File main.py:4 from pipeline import Pipeline
File pipeline.py:6 from clingo.control import Control
ModuleNotFoundError: No module named 'clingo'"
Please upload the missing files so that we can run your code.
Best Regards Navdeep

Hello,
I am retracting my request. I just had to install the clingo software in python to get the above error fixed.
Best
Navdeep


Improved Visual Cohesion on About Us Page

Fixes #107 : Change About Us page design
I have enhanced the visual cohesion between the background of the About Me page and the foreground content by:
-Adding a radial gradient to the background of the main container.
-Applying a blurry backdrop-filter to the container to create a subtle blur effect.

This change helps unify the background and foreground elements, creating a smoother and more cohesive design. The effect is subtle but noticeable, and it improves the overall aesthetics of the page without being distracting.
Testing:
-Verified the visual changes on Chrome and Firefox.
-Ensured the layout remains responsive and the effect renders properly on desktop and mobile.

I had another go of it, with a more subtle background, and some minor style changes:

Please note that I have added a background color to body for smaller screens. The following image illustrates what happens without said change. The problem seems to arise because of how the background photograph is implemented, however, I could not find a satisfying solution.

Oh I see, it seems that pr #121, which is already merged to main, fixes #107 (that the present pr tries to solve). The two prs are not compatible. It's one or the other, as far as I can tell. I have no problem with you choosing the other solution over mine. It was a fun exercise!

Thank you, @Virtual4087.
See pr #122.

@admwln Done I'll be closing this pr. Thanks for the help.


update getting-started-guides/coreos to v1

a loose end in #5799
@bgrant0607 @nikhiljindal @krousey

LGTM

@k8s-bot test this please

Please look more closely.
v1beta1 != v1 schema.

ah...Forgot the metadata block..Sorry.

Fixed. PTAL, thanks.

LGTM

Needs 2nd LGTM @quinton-hoole

Trying @lavalamp for a 2nd LGTM

lgtm, ok-to-merge.  Sorry - I just saw this one.  Feel free to assign to me in future to get my attention.


While waiting for account to be ready, show alert messages at each bit of progress

(Account generation complete + after each tx.)

@marcelomorgado So far this step has taken longer than early testers expected, so I think this would be a good quick win

The longer-term solution will be something like this - https://github.com/tasitlabs/tasit/issues/188 - but having a good sense of timing for each step will help with that

The UX is sooo much better now after this change

Great!


Fixed typo in auth.go

Description
Fixed typo in auth.go
Fixes #156
Type of change

Bug fix (non-breaking change which fixes an issue)
Documentation update

Testing steps
N/A

Codecov Report

Merging #157 into master will not change coverage.
The diff coverage is 50.00%.

@@           Coverage Diff           @@
##           master     #157   +/-   ##
=======================================
  Coverage   49.21%   49.21%
=======================================
  Files          12       12
  Lines        1024     1024
=======================================
  Hits          504      504
  Misses        461      461
  Partials       59       59

Impacted Files
Coverage Δ

server/auth.go
47.94% <50.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b49ed2a...60346c4. Read the comment docs.


Inlining breaks query parameters in URLs

Hi there,
first of all thank you for this library. I wanted to use this to replace juice in NodeJS. Unfortunately, I might have discovered a bug.
Version: css-inline = "0.8"
const HTML: &str = r#"

    <title>Test</title>

Big Text

    Link

"#;

fn main() -> Result<(), css_inline::InlineError> {
    let inlined = css_inline::inline(HTML)?;
    // Do something with inlined HTML, e.g. send an email
    println!("{}", inlined);
    Ok(())
}

Actual output:

    <title>Test</title>

Big Text

    Link

The problem:
The query parameters of the URL are altered to a state where they are not correct anymore.

Correct:  https://example.org/test?a=b&c=d
Wrong: https://example.org/test?a=b&c=d"

I'll try to fix this but I'm unsure if I find the right spot.

:cry:
Seems like the error is in the kuchiki library itself.
This test fails like explained above:
#[test]
fn check_kuckichi() {
    let html = r#"

    Link

"#;
    let document = kuchiki::parse_html().one(html);
    let mut target= Vec::new();
    document.serialize(&mut target);
    assert_eq!(String::from_utf8_lossy(&target).to_string(), r#"
    Link

"#);
}

I am wondering whether it originated from deeper levels like ‘html5ever’, but nonetheless ‘kuchiki’ became unmaintained and we have to replace it at some point, so in this case we need to keep this behavior in mind

For reference - it happens here inside html5ever

However, it is probably escaped because of this:

Attribute values are a mixture of text and character references, except with the additional restriction that the text cannot contain an ambiguous ampersand.

But I think it is fine in our case as we just need to pass things as they are in the original document.

I re-implemented the HTML serialization step on our side and dropped escaping, which fixes the problem. Hopefully, the fix will be released soon - I also switched the HTML tree representation from kuchiki to an in-house one, so I'd like to run more tests & fuzzing on it before release.

This fix was incorrect, and introduced a security vulnerability by transforming safely escaped HTML into unescaped output. Thanks @Stranger6667 for a very quick resolution following my private disclosure of this vulnerability. #188 was reverted in css-inline 0.10.0 (#197), and 0.9.0 was yanked from PyPI and crates.io.
The original report here is not a bug. It’s completely fine to convert  to . These have identical parse trees:

In , the & is correct HTML syntax that represents a & in the string value of the attribute. The & is not interpreted as part of the URL and isn’t sent to the server or anything like that.

In , the & is incorrectly underescaped, but the HTML5 parsing algorithm has a fallback branch to tolerate this error, resulting in the same string value.

In general, HTML escaping is necessary to safely serialize HTML attributes, because otherwise certain special characters like " would be unrepresentable. Although HTML5 parsers are required to be tolerant of certain underescaping errors, the exact conditions under which it’s safe to rely on this tolerance are complicated (within a double-quoted attribute, underescaping " is never okay, and underescaping & is only sometimes okay). So in practice the only reasonable strategy for a safe HTML generator is to escape special characters in attributes.
If you have a tool that interprets  as a URL containing &, that tool is wrong and needs to be fixed to use a real HTML parser.

First of all, thank you for the feedback and thanks for taking the time.


BD-1037:Compliance Issues in ThunderClientLibraries

Copyright scan failure
Commit: eb6734414a76e6f227046c93b2acaa9e80d30bb3
Report detail: https://gist.github.com/ec890d67c613c58ff97f9f0f1e1d278b

Blackduck scan failure details
Summary: 0 violations, 0 files pending approval, 4 files pending identification.

Protex Server Path: /home/blackduck/github/ThunderClientLibraries/111/rdkcentral/ThunderClientLibraries

Commit: eb6734414a76e6f227046c93b2acaa9e80d30bb3

Report detail: gist

Copyright scan failure
Commit: 9315fb0d31b6aeb228013f76cdf33bccbcdb7a0b
Report detail: https://gist.github.com/b92d914078a1be3f6a26530ab153bca1

Blackduck scan failure details
Summary: 0 violations, 0 files pending approval, 4 files pending identification.

Protex Server Path: /home/blackduck/github/ThunderClientLibraries/111/rdkcentral/ThunderClientLibraries

Commit: 9315fb0d31b6aeb228013f76cdf33bccbcdb7a0b

Report detail: gist

Copyright scan failure
Commit: ff31e1d7e18ae80d7de1172ffeb11b9da6fb8de0
Report detail: https://gist.github.com/0a4bf7681f7d1e41742e0c1ae886ea81

Blackduck scan failure details
Summary: 0 violations, 0 files pending approval, 4 files pending identification.

Protex Server Path: /home/blackduck/github/ThunderClientLibraries/111/rdkcentral/ThunderClientLibraries

Commit: ff31e1d7e18ae80d7de1172ffeb11b9da6fb8de0

Report detail: gist

Blackduck scan failure details
Summary: 0 violations, 0 files pending approval, 4 files pending identification.

Protex Server Path: /home/blackduck/github/ThunderClientLibraries/111/rdkcentral/ThunderClientLibraries

Commit: ff31e1d7e18ae80d7de1172ffeb11b9da6fb8de0

Report detail: gist

Sorry Bhargavi, not sure why I didn't see this earlier:

Change the RDK copyright in top level CMakelists.txt.  (I think this is the last RDK one.)
Please do a grep for "this component's license file" and change to "this component's LICENSE file" (just a change of case).  This is in about 20 files.
Thanks,
Martin

Copyright scan failure
Commit: b07d4aec0bb4bdb5895ab74a22f80b99a2e301d4
Report detail: https://gist.github.com/d7f55b818a2207a56f3741795d153fa8

Thanks Bhargavi.  I updated the BLDKCMF ticket to put Metrological questions in bold.


Semantics of inclusion of sender's key confusing

From<EMAIL_ADDRESS>on July 06, 2014 23:35:24
Steps to reproduce:
1. Send a message to a recipient and delete the sent message
2. Have the recipient delete her public key, but not the received message
What happens? The recipient now has a copy of the message that the sender can decrypt. The sender does not have a copy of the message TO decrypt, but could decrypt the recipient's copy.
Some thoughts about the right semantics:
A. E2E used with Gmail (or other services with mail-semantics)
If a user is using E2E with Gmail, they know that they are identifiable as the sender of the message. It is therefore appropriate to maintain traditional email semantics by including a readable copy of the message.
It would be better, however, to do so by BCCing the sender on the message, and including a PKESK under the sender's key with this BCCed message -- but not with the message sent to the recipients.
(Otherwise, the situation is confusing: If the recipient deletes her public key, but not the message, and the sender deletes the sent message, but not his public key, the recipient has a copy of the message that the sender can decrypt.)
B. E2E in other browser contexts (cut-and-paste)
If a user is using E2E in another browser context, it should be assumed that they are not using a system with mailbox semantics. The message should not be encrypted to the sender's public key, without explicit indication that this is the desired behavior.
(In particular, consider a potential source anonymously emailing a reporter; it is very undesirable -- and potentially dangerous -- for the sender to be able to decrypt the message. At the moment, because the recipient's public key is included in all PKESK packets, anonymity is violated.)
Original issue: http://code.google.com/p/end-to-end/issues/detail?id=103

Closing this - we're staying with the current behavior. Reasoning - it's the most expected one from an OpenPGP implementation.


Save policy using PPO1 baseline

Hi!
Does anyone know how to properly save a learned policy obtained using the PPO1? There is not any save method implemented, and a simple dump method with pickle won't work.
Thanks!

Issue #74 and this comment might be helpful.


ENH: Fix Node.js warnings linked to artifact download GitHub actions

Fix Node.js warnings linked to artifact download GitHub actions: transition to actions/download-artifact@v3.
Fixes:
python-build-workflow / publish-python-packages-to-pypi
Node.js 12 actions are deprecated. For more information see:
https://github.blog/changelog/2022-09-22-github-actions-all-actions-will-begin-running-on-node16-instead-of-node12/.
Please update the following actions to use Node.js 16: actions/download-artifact@v2

raised for example in:
https://github.com/InsightSoftwareConsortium/ITKTextureFeatures/actions/runs/3832886629
Cross-referencing
https://github.com/actions/download-artifact/pull/134
Left behind in commit 6e81bbd.

In the future it would be helpful to test changes via an external module fork to demonstrate that desired functionality is accomplished. I've typically used a development branch in my ITKSplitComponents fork with the latest workflow PR commit hash to verify changes. I will look into documenting that process in this repo

Thanks Tom. The thing with these warnings is that unless one has a look at the action summary page, and scrolls down past the component graph, they still go unnoticed: i.e. the builds themselves are not expect to fail, or warnings/failures be reported in the dashboard. Not sure if there is a way to tell GHA to report a build as failed if actions report warnings.

I've typically used a development branch in my ITKSplitComponents fork with the latest workflow PR commit hash to verify changes

I've seen the effort. Maybe a more sustainable and automated solution could be adopted, maybe with a remote repository that has minimum contents, much like the one the ITKModuleTemplate cookiecutter creates by default.

The "upload to PyPI" step is difficult to test because it is only expected to run for tagged releases

I guess that can be automated and have it tested automatically as well.
But I understand that all of the above may require work and may not be easily done.

@jhlegarreta I've opened #48 to track, feel free to add additional discussion on testing there


.Net: Example44_MultiChatCompletion needs to be rewritten

Describe the bug
The example has the following issues:

It is broken into multiple methods and so is not copy/paste friendly
It uses the IChatCompletion directly, which is not our preferred pattern
It doesn't use a prompt as the input for the example

All .Net issues prior to 1-Dec-2023 are being closed. Please re-open, if this issue is still relevant to the .Net Semantic Kernel 1.x release. In the future all issues that are inactive for more than 90 days will be labelled as 'stale' and closed 14 days later.


TM Processing L0: 3,25,2 hk-maxi

2022-07-10 16:31:22,515 MergeConflictWarning: Cannot merge meta key 'CHECKSUM' types <class 'str'> and <class 'str'>, choosing CHECKSUM='ToDBWm99TmAATm97'
2022-07-10 16:31:22,516 MergeConflictWarning: Cannot merge meta key 'DATASUM' types <class 'str'> and <class 'str'>, choosing DATASUM='1058969377'
2022-07-10 16:31:22,524 Error processing file /data/stix/out/test/bsdfull/L0/3/25/2/solo_L0_stix-hk-maxi_0692323200_V01.fits
Traceback (most recent call last):
File "/home/nicky/STIXCore/stixcore/processing/L0toL1.py", line 69, in process_type
files = processor.write_fits(l1)
File "/home/nicky/STIXCore/stixcore/io/fits/processors.py", line 762, in write_fits
prod = prod + existing
File "/home/nicky/STIXCore/stixcore/products/product.py", line 495, in add
data = vstack((self.data, other_data))
File "/home/nicky/STIXCore/venv/lib/python3.8/site-packages/astropy/table/operations.py", line 651, in vstack
out = _vstack(tables, join_type, col_name_map, metadata_conflicts)
File "/home/nicky/STIXCore/venv/lib/python3.8/site-packages/astropy/table/operations.py", line 1393, in _vstack
col_cls = _get_out_class(cols)
File "/home/nicky/STIXCore/venv/lib/python3.8/site-packages/astropy/table/operations.py", line 88, in _get_out_class
raise ValueError('unmergeable object classes {}'
ValueError: unmergeable object classes ['Column', 'Quantity']
2022-07-10 16:31:22,526 unmergeable object classes ['Column', 'Quantity']
2022-07-10 16:31:23,169 VerifyWarning: Card is too long, comment will be truncated.
2022-07-10 16:31:23,315 VerifyWarning: Card is too long, comment will be truncated.
2022-07-10 16:31:23,317 VerifyWarning: Card is too long, comment will be truncated.
2022-07-10 16:31:23,521 VerifyWarning: Card is too long, comment will be truncated.

i had to inject the x4 % unit for the cpu_load into older IDB version for uniform data type over entire mission

note: % will become  Unit(dimensionless with a scale of 0.01) after FITS in/out


Update AbstractData.php

Ability to set default return values when a property is not available in the probe results

Thanks!


Why is the available memory smaller than my configuration?

at machine type,i config mem: 60G,when i call this  http://master_ip:8088/ws/v1/cluster/nodes to get the available resources of my workers, i got this 46080
another worker has 30G mem, but 15360
where is my memory

Some memory is reserved for PAI service

PAI  eat 15G ?

It's for prod configuration, anyway, it's a opensource project, you could change it as wanted
https://github.com/Microsoft/pai/blob/a01d6b2606d58b6b745aaf05fb1ca7856b52b60a/src/hadoop-node-manager/deploy/hadoop-node-manager-configuration/nodemanager-generate-script.sh#L66

thank you for response


Improve examples

I'm usually using my fork of this package with gin but now I'm experimenting with ent and graphql and trying to use this package and net/http.
The examples are spaghetti code at its finest.
Make them examples readable, do not make the reader jump around varioud subdirectories, hiding how it works.
Add an example for keycloak, you know the software you can self-host and not hand over your user data to 3rd parties unlike octa, auth0, conginto and especailly azure.
Jank the core oidc package OUT of internal so people can import it and keep up to date with the changes.
JFC! Unstarred this repo because I'm so annoyed by this.

Thank you for the amazing feedback @idc77! I will take it into consideration for future improvements.


WIP: Generate span model from proto

Signed-off-by: Pavol Loffay<EMAIL_ADDRESS>

@annanay25 I want to ask you two things

The generated span model class in this repository is different than the original. You have mentioned that you were able to generate exactly the same class. How did you do it? The code I generated contains references to gogo and it requires to add https://github.com/QubitProducts/gogoproto-java to java classpath otherwise Java code does not compile.
query.proto generates gRPC services without problems. However import "protoc-gen-swagger/options/annotations.proto"; is causing Java compilation error. Any idea how to fix it? Should we also generate grpc.gateway.swagger code?

/home/ploffay/projects/jaegertracing/trace-dsl/proto/src/main/java/io/jaegertracing/api_v2/Query.java:[9839,50] package grpc.gateway.protoc_gen_swagger.options does not exist


[SPARK-41378] Support Column Stats in DS v2

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I wish the RAPIDS Accelerator for Apache Spark would [...]
Spark 3.4 has supported Column Stats in DB v2. We should check if we support it, either.
Describe the solution you'd like
A clear and concise description of what you want to happen.
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context
Add any other context, code examples, or references to existing implementations about the feature request here.
https://github.com/apache/spark/commit/96843850b4

a follow up change can also be included for this change: https://github.com/apache/spark/commit/e43b4be77a. Add it here.

We do not overwrite relations so we should be ok.


Option to get spec path and file name

Hello,
It would be great if there was a way to extract file names and paths of recently run tests. It would allow us to make reporters with pages to different tests. So for example you could run [spec1.js, spec2.js, spec3.js] in protractor and then you would get not only results for each test, but also its filename and path, and based on that you could generate list of specs that would redirect you to each individual test result.
Currently I found that the only way is to create 'global' describe for each test with its filename as description, but this requires testers to follow this structure.

We could probably add this to the resultJsonOutputFile without too much trouble.

Please... with sugar?

+1 here, this would be amazing


go: fix the merge race

Fix the bad merge of the last two PRs.
/assign @htuch

see the second bash result https://dev.azure.com/cncf/udpa/_build/results?buildId=22254, that should fail the check.

@lizan This is weird. Why is bazel query crashing on Azure? What do I need to pass? It works fine locally.

OK, once TEST_TMPDIR is defined it works. PTAL.

just posted #16, rm go/udpa and git add to track file addition/removal too.


[Torg Eternity] V.3.2

Changes / Comments
Tested on Firefoxx & Chrome

Fixed tab buttons so they also display properly on Firefox (Archange, with minor adjustments from Anduh)
Enlarged Logo to be readable
minor CSS/HTML cleanup
Changelog update

Roll20 Requests
Comments are very helpful for reviewing the code changes. Please answer the relevant questions below in your comment.

[x] Does the pull request title have the sheet name(s)? Include each sheet name.
[x] Is this a bug fix?
[ ] Does this add functional enhancements (new features or extending existing features) ?
[x] Does this add or change functional aesthetics (such as layout or color scheme) ?
[ ] If changing or removing attributes, what steps have you taken, if any, to preserve player data ?
[ ] If this is a new sheet, did you follow Building Character Sheets standards ?

If you do not know English. Please leave a comment in your native language.

You've been busy!

You've been busy!

Yeah, I've been on a roll lately :grin:


"schedule is undefined" in d3 3.5.7

Hi!
I just discovered a bug in the latest version (3.5.7) of d3. Version 3.5.6 is not affected.
Introduced here the function definition of "schedule" happens inside an if block, is not hoisted to the top and can thus not be used as a parameter for d3.timer(). Actually, function definitions inside an if are prohibited in strict mode.
Seems to work in Chrome, but Firefox (38) throws an error: ReferenceError: schedule is not defined

Looking.

This is only a problem with the non-minified JavaScript, too.
This feels like a Firefox bug to me. I understand that the functions are hoisted, but this shouldn’t cause them to be undefined. Still, the fix is trivial.

Fix staged in #2614. I will push a release shortly.

Cool, thanks!
Yes, that was the non-minified d3.js. But then again, another minifier might treat it differently :)
Anyway, thanks for the quick resolution :sparkles:

On 10 Nov 2015, at 17:36, Mike Bostock<EMAIL_ADDRESS>wrote:
Closed #2613 via 959da21.
―
Reply to this email directly or view it on GitHub.

This feels like a Firefox bug to me. I understand that the functions are hoisted, but this shouldn’t cause them to be undefined.

It's not a bug per se: the behavior in cases like this used to be completely unspecified, and browsers just did different things. The ES6 spec finally contains a specification, if only in an annex. Note that the difference is that in SpiderMonkey, the function is not hoisted right now, whereas it is in other engines. Way more details than you're probably interested in can be found here: http://kangax.github.io/nfe/#expr-vs-decl. Implementing the spec'ed behavior is tracked in bug 1071646.

Thanks for the additional info; I wasn’t aware this was a recent standardization. I changed the release notes to be less accusatory. (Sorry!) I agree we should avoid conditional function declarations at least until the spec’d behavior is universally available.


Use currentColor for better CSS overriding

Without this, customizing the fill color of the next and previous arrows requires duplicating the background-images (totaling 632 chars)

:+1: Good catch! Thank you very much for the contribution :)


Added Fulwood Academy Sixth Form

Added Fulwood Academy Sixth Form

@LouisHacking I am sorry for a delay with processing your request. Could you please give us a reference to any IT-related long-term (longer than 1 year) course which your Academy offers to students? I failed to find it among courses at http://www.fulwoodacademy.co.uk/sixth-form/courses/
It also helps a lot if you can somehow prove that the email addresses in fulwoodacademy.co.uk are available to the students and teachers only, not alumni or other people who do not study or teach. It can be a page with the Academy policy or something like that.
Thank you!

@LouisHacking I am sorry, this PR remains unanswered for a while, and I close it without merging. If you have any updates regarding the info I asked about in the previous comment, please leave your comment and reopen the request.
If you have very limited number of IT-related courses or they are actually short-term, you may consider other options also:

Use trial version of JetBrains tools for the courses.
Apply for Classroom license by email to<EMAIL_ADDRESS>
Thank you for your understanding.


Use generic targets rather than local mock proxy

Don't use a local pingstatus-mock proxy or provide that proxy and install first.

I am trying to replicate this in apigeeX with hello-world api , api deployment works fine but product and app creation fails.
I have created a code repo and placed apiproxy and resource folder in the repo at root level  without the parent folder.
--- apigee-config-maven-plugin:2.1.2:apps (default-cli) @ hello-world ---
[WARNING] The artifact xml-apis:xml-apis:jar:2.0.2 has been relocated to xml-apis:xml-apis:jar:1.0.b2
[INFO] ************************************************************************
[INFO] Apigee App
[INFO] ************************************************************************
[INFO] Config file org/developerApps.json not found.
[INFO] No developers apps found.
The file resources/edge/org/developerApps.json exist .


group_manifests: create plugin for group manifests

group_manifests: create plugin for group manifests
Add a group_manifests plugin. For now, it fails if asked to create
a group of manifests.
If asked to create a single manifest, it pulls the x86_64 image manifest
from a repository passed as an argument. It then re-uploads the image
manifest to a new location for each image in the tag_conf. It returns the
image manifest.
Signed-off-by: Mark Langsdorf<EMAIL_ADDRESS>

This code is most likely wrong, but it's close enough for review.
I suspect that this code:
            digest = worker_digests[0]['digest']
            tag = worker_digests[0]['tag']
            repo = tag.split(':')[0]
            for image in self.workflow.tag_conf.images:
                image_tag = image.to_str(registry=False)

                url = '{}/v2/{}/manifests/{}'.format(registry, repo, digest)

                response = requests.head(url, verify=not insecure, auth=auth)
                response = requests.get(url, verify=not insecure, auth=auth)

should use the image name in place of the digest, but I'm not 100% sure.
I can't get this code to fail if there's an amd64 platform in the annotations
        for platform in all_annotations['worker-builds']:
            if self.goarch.get(platform, platform) == 'amd64':

I suspect that is not the desired behavior, but I don't understand what goarch is for.

I suspect that this code:
        digest = worker_digests[0]['digest']
        tag = worker_digests[0]['tag']
        repo = tag.split(':')[0]
        for image in self.workflow.tag_conf.images:
            image_tag = image.to_str(registry=False)

            url = '{}/v2/{}/manifests/{}'.format(registry, repo, digest)

            response = requests.head(url, verify=not insecure, auth=auth)
            response = requests.get(url, verify=not insecure, auth=auth)

should use the image name in place of the digest, but I'm not 100% sure.

Why? I think the assumption can be made that the worker pushed the same image for all the tags, and the digest that the worker returned is the reliable way to get exactly that image for that architecture. As @lcarva said, I think the head can be removed and this can be pulled out of the loop.

What goarch is for: manifest lists describe platforms in terms of the strings used by the Go language to describe operating systems (GOOS) and architectures (GOARCH). We're using platform names such as 'x86_64' to say GOOS=linux, GOARCH=amd64. So far, the GOOS is assumed to be 'linux', but to work out the GOARCH we need a translation dictionary to change 'x86_64' into 'amd64'. Hopefully other platform names will be the same as the GOARCH names. FWIW I just submitted a docs update to describe how this translation dictionary will be configured: https://github.com/projectatomic/osbs-docs/pull/31

tested with brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/mlangsdo/buildroot:eng-rhel-7-docker-candidate-94693-20170816174925, which is a build of pkgs.devel.redhat.com/rpms/osbs-buildroot-docker@private-mlangsdorf-incremental
that uses:
https://github.com/mlangsdorf/atomic-reactor.git@arrangement4
https://github.com/mlangsdorf/osbs-client.git@arrangement4
build logs are here:
https://osbs.qa.engineering.redhat.com:8443/console/project/osbs-stage/browse/builds/helloworld-container-master-9bc2a/helloworld-container-master-9bc2a-77?tab=logs
please merge thanks.

@mlangsdorf, is this OK to merge? I think it is, just checking.

IIUC https://github.com/projectatomic/atomic-reactor/pull/778 supersedes this PR

I'd like to be able to perform testing for arrangement version 4 which requires group_manifests, but does not require group=True to work.

Draft release notes updated.


Can't remove keys defined in values.yml

Since the introduction of deep merging (#1620), it's now not possible to remove keys from values.yml entirely.
For example, the telegraf values has a default entry for single.config.inputs.influxdb (see: L340). There's no way to completely remove this key through --set or a override file.
Telegraf values.yml
single:
  config:
    inputs:
      influxdb:
        urls:
          - "http://influxdb-influxdb.tick:8086/debug/vars"
        timeout: "5s"

My telegraf-overrides.yml
single:
  config:
    inputs:
      influxdb: null # or something?

Yeah, this appears to be true. Any suggestions about how we might support that?

A few options come to mind:

A flag to indicate the values you're supplying replace the values from the chart. helm install -f my-values.yml --overwrite
A special value in the yaml to indicate not to merge, perhaps a null. e.g. Null means delete this key

Meta: the reason this is a problem is because Telegraf doesn't have an explicit enabled property you can set, so the presence of a value is significant.
We could work around this by adding our own enabled key under influxdb, and check for the presence that in the templating and not write out the node; but this puts the onus on each chart to get it right.
More meta: I've also been thinking about whether we should be trying to generate config files from YAML in the first place. Several charts (the Influx stack in particular) allow you to specify configs in YAML which get translated into TOML, this translation is quite complicated and I suspect we're losing some information in translation. Each chart is having to do this itself. It might be better to just not try to do this at all, have the ConfigMap contain non-generated TOML which you can just pass your own in wholesale. Most of the time I already have a config file I want to use, and translating it to YAML is arduous (though I appreciate being able to abstract away complicated configs).

Prior art for my last little ramble: grafana.ini

I wouldn't be opposed at all to an --overwrite flag. That makes sense.
Some work did make it into Helm 2.2.0 which will allow subcharts to be switched on and off based on conditions. But it will likely take the charts repository a while to catch up on those.

Hey guys, just coming to this now. We were just brainstorming about this problem in Slack Today. And just as I was about to create an issue, I looked and found this one 😄
So I prefer the null option, over --overwrite, because that could change more than we need. The immediate problem I see with an --overwrite flag is that would change the deep nesting behavior of all the keys/values from #1656 (helm install some/chart -f values1.yaml -f values2.yaml --set foo.bar=baz. I would think we want to be able to target specific keys, otherwise lots of breakage may happen. Whereas we could just add one check for a null value during the Helm deep nesting logic, and in that case nix the key.

I also like the null option and  using it with helm init --set would allow advanced users to  completely remove keys (e.g. the default nodeSelector that specifies linux) and have more control of the Tiller Deployment manifest.

@scottrigby So I was looking at the code as it stands without your PR. It looks like ~ indicates null in YAML. I was able to overwrite any key using that notation. However, with that said, it won't actually delete the key, it will just set its value to null. So I think your PR is a better option because it actually deletes the key from the map

Yeah, I too prefer an explicit null.

If ya'll agree, could we add this to 2.6 under the "stability and security" heading?

Yes, I agree and already labeled the PR as such 😄

Closed in #2648


missing [] and {}

missing [] and {}
Actual behavior:
the key combination no longer works on the console to type {} or []
Expected behavior:
How to reproduce it (as minimally and precisely as possible):

What are your environment parameters?

DeFi Wallet Version: DeFi Wallet 2.3.5
Operating System (OS): macOS Big Sur
– Current free storage capacity: 400 GB

Anything else we need to know?

Hi @izzycsy, still not working?
I have opened up two issues about this case already, they were closed because of no updates. Last one:
https://github.com/DeFiCh/app/issues/543

Hi, still not working in 2.3.4 and 2.3.5!

On 17.05.2021, at 13:28, tame81 @.***> wrote:
Hi @izzycsy https://github.com/izzycsy, still not working?
I have opened up two issues about this case already, they were closed because of no updates. Last one:
#543 https://github.com/DeFiCh/app/issues/543
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub https://github.com/DeFiCh/app/issues/859#issuecomment-842247131, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEJCSNZVKR45O67SMOLTPGDTOD4U7ANCNFSM4464HSRQ.

Hi @tame81 , thank you, the team is aware. It could be the German keyboard that's giving you the issue.
Hi @skorilo , may I know what commands were you using?
When I try the following it does work.
listaccounts {} false false true
[
  {
    "key": "a9149xxx@0",
    "owner": "dRmyxxx",
    "amount": "0.00xxx@DFI"
  },

Same problem for me: Mac OS with German keyboard. Problem is, that i can copy the {} into the node shell but can't type it into the shell directly. The key combination is the "option" key and number "8" / "9" key.

@izzycsy: I know it's an issue with the German keyboard (on the American one you can access curly braces and angle brackets without having to use the option key).
There no such problem in any shell or any app on the Mac but the DeFi Wallet. This should be fixed, it's very inconvenient to copy those special characters over to the CLI.

Hi @skorilo @tame81 @sladig , I understand the inconvenience, the team is aware however we've to prioritise urgent bugs such as users not being able to load their app etc., thanks for your patience.
v2.4.2 is out, this is a mandatory update. Please read through the release notes before updating. This comment is to notify you about the update.

As with v2.5.1, the problem is still not fixed.
There's no such problem in any shell or any app on the Mac but the DeFi Wallet. This should be fixed, it's very inconvenient to copy those special characters over to the CLI.

Hi @skorilo @tame81 @sladig , update v2.6.0 is out, this is a mandatory update.
Please read through the release notes before updating. This comment is to notify you about the update.

Thanks @izzycsy for the info! The issue has not been solved though.

Hi @skorilo @tame81 @sladig update v2.6.1 is out.
Please read through the release notes before updating. The above feature / issue is not included in this update yet, this comment is to notify you about the update.

Hi @skorilo @tame81 @sladig , update v2.6.2 is out.
Please read through the release notes before updating. This comment is to notify you about the update.

Hi @skorilo @tame81 @sladig , update v2.6.3 is out.
Please read through the release notes before updating. This comment is to notify you about the update.

Hi @skorilo @tame81 @sladig , update v2.7.0 is out, this is a mandatory update. Kindly backup wallet before updating.
Please read through the release notes before updating. This comment is to notify you about the update.

Hi @skorilo @tame81 @sladig , update v2.7.1 is out, this is a mandatory update. Kindly backup wallet before updating.
 Please read through the release notes before updating. This comment is to notify you about the update.


Won't go past boot screen

I have it wired correctly and the code compiles with no errors, however it only shows the Retro-watch boot screen and never continues to the clock.

did u got the answer?


Use in-worker ngx.semaphore instead of resty.lock

For slot_cache, I think the in-worker ngx.semaphore is enought.

Add resty.lock here is just to avoid several coroutine trying to init slot-node cache which is unnecessary when startup. There is no issue even lock is removed.


Add fast access function

Currently we check for the number of arguments == dimension in the access operators (e.g. operator() etc.
This has a substantial performance hit for xarray containers where the dimension is not known at compile time and prevents effective automatic vectorization and such things (performance hit on simple for loops ~17x ...
Therefore we should either add a new fast access method, or remove this check.

On the same (even more important note) we currently use the element[...] function in the xindex_stepper.
Therefore, when we use this stepper on an xarray, we perform a check on the dimension every iteration step -- which is wrong as we have perfect knowledge in our stepper.

I confirm that I have also measured a slowdown ranging from 20X to 5X compared to c-style array access for 1d and 2d arrays. I think that xarray and xtensor access operations should really match plain arrays performances for common accessors and that an overhead should only occur when extra functionalities are needed.
Also it seems that for xtensors, there should not be any dimension check at runtime. However, the access penalty is the same as with xarrays.

Actually the compiler should be able to optimize this check for xtensor (that is what @wolfv noticed on our side).
However, the access operator in xtensor brings the broadcasting functionality so the extra check and the extra cost. For instance, you can write code like:
auto f = a + b;
double d = f(i, j, k);

and it works even when a and b have different dimensions (based on the broadcasting rules). The dimension check is here to drop extra indices in operator(), so the following invariant is always satisfied:
(a + b)(i, j, k) = a(i, j, k) + b(i, j, k);

We can add an access method that does not perform this check, however it won't be compatible with broadcasting.

yes, I think that it would be a good solution in order to cover all the situations where nd-arrays are needed but not broadcasting. For the moment, one can avoid the cost of broadcasting rules for 1d arrays (using storage or data) but there is no (direct) solution for nd-arrays.


Command for the MSA Transformer in the Variant Prediction example results in a runtime error

Bug description
Running the example command for the MSA Transformer in the Variant Prediction example in https://github.com/facebookresearch/esm/tree/main/examples/variant-prediction results in a runtime error "Received unaligned sequences for input to MSA, all sequence lengths must be equal".
Reproduction steps
python predict.py \
--model-location esm_msa1b_t12_100M_UR50S
--sequence HPETLVKVKDAEDQLGARVGYIELDLNSGKILESFRPEERFPMMSTFKVLLCGAVLSRVDAGQEQLGRRIHYSQNDLVEYSPVTEKHLTDGMTVRELCSAAITMSDNTAANLLLTTIGGPKELTAFLHNMGDHVTRLDRWEPELNEAIPNDERDTTMPAAMATTLRKLLTGELLTLASRQQLIDWMEADKVAGPLLRSALPAGWFIADKSGAGERGSRGIIAALGPDGKPSRIVVIYTTGSQATMDERNRQIAEIGASLIKHW
--dms-input ./data/BLAT_ECOLX_Ranganathan2015.csv
--mutation-col mutant
--dms-output ./data/BLAT_ECOLX_Ranganathan2015_labeled.csv
--offset-idx 24
--scoring-strategy masked-marginals
--msa-path ./data/BLAT_ECOLX_1_b0.5.a3m
Logs
Traceback (most recent call last):
File "predict.py", line 241, in
main(args)
File "predict.py", line 167, in main
batch_labels, batch_strs, batch_tokens = batch_converter(data)
File "/home/ubuntu/miniconda3/envs/msa_trans/lib/python3.7/site-packages/esm/data.py", line 328, in call
"Received unaligned sequences for input to MSA, all sequence "
RuntimeError: Received unaligned sequences for input to MSA, all sequence lengths must be equal.

@rmrao Roshan, could you take a look at this please?

I think the failure is happening now due to this code change. variant-prediction now removes lowercase letters from the input MSA, which is consistent with contact_prediction.ipynb. The A3M spec on one website at least says lowercase letters correspond to insertions, which I believe means they should be removed.
But the lowercase letters in the demo a3m files seem to be different. This error shows that removing lowercase letters from BLAT_ECOLX_1_b0.5.a3m yields sequences with different lengths. I wonder which tool was used to create BLAT_ECOLX_1_b0.5.a3m? Should all the MSA reading code be standardized around that tool's definition of an a3m file?


add filename extension

Changes default name to have .pt.  Fixes CO-850.

Forgot to mention, but do we want the latest symlink to also have an extension?

Forgot to mention, but do we want the latest symlink to also have an extension?

Its in the PR -- yea I think we do so it looks like a checkpoint file


feat: add support for AzureSQL to AzureSQL

 Problem
In order to support AzureSQL to AzureSQL masking, we need to add type mappings from Azure SQL data types to ADF data types.

 Solution

Wrote a migration script that will add type mappings from Azure SQL data types to ADF data types in adf_type_mappings table.
Excluded the following types since we cannot mask them - table, cursor, sql_variant, hierarchyid, rowversion

 Testing Done
Verified the mappings via Projection:

I'm aware PrePush Jira checks are failing because I did not add the JIRA ticket number in the PR title. I'm not sure if we should add it given that previous PRs and commits on this repo did not have the JIRA ID.

Closing this PR as we have created a new PR (#24) that consolidates all Azure SQL pipeline code.


Push notifications Page redirect

Hi all,
I used FCM plugin for push notifications in ionic 2
Below my code
declare var FCMPlugin;
//Push Notifications
  if(typeof(FCMPlugin) !== "undefined")
  {
    FCMPlugin.getToken(function(t){
      console.log("Use this token for sending device specific messages\nToken: " + t);
    }, function(e){
      console.log("Uh-Oh!\n"+e);
    });
     FCMPlugin.onNotification(function(d){
      if(d.wasTapped){
        // Background receival (Even if app is closed),
        //   bring up the message in UI
        console.log(d);
       this.nav.setRoot(page);
      } else {
        // Foreground receival, update UI or what have you...
        console.log("Push Notification", d);
        console.log("you have new notifications");
      }
    }, function(msg){
      // No problemo, registered callback
    }, function(err){
      console.log("Arf, no good mate... " + err);
    });
  }

It's work perfectly.My app received notifications.But when i click notifications does not redirect to page.
Kindly Advice me,
Thanks & Regards.

https://ionicframework.com/docs/v2/native/push/

Thanks for ur reply.any other way to page redirection in when I click notifications.this.nav.setRoot(page); cannot wrk in my app.and also I tried this.nav.push(page) it,s wrk perfectly an page navigation.but I cannot wrk an push notifications click

This might be related to your payload, see: https://github.com/phonegap/phonegap-plugin-push/issues/1609
Or, that "wasTapped" isn't working. My code says:
if (data.additionalData.foreground) instead of that if(data.wasTapped)
And anyway, this issue is a support request, and my best guess it would be closed soon, so you should try the forum.
Good luck

Hello! Thanks for opening an issue with us! As this seems like more of a support question i will urge that you ask this question on our forum or on our slack channel. Thanks for using Ionic!


Migration issue for Spring Security. Application start up failure

Hello Team,
I am using the below documentation for migration to the spring security and use converter interface through Autowiring instead of xsuaa packages as described in the sample provided in documentation.
I have checked the xsappname configuration and tried it in both ways: though VCAP_SERVICES env variable as well as tried adding in application .properties file.
But I am facing the below error while configuration.
Also tried using the custom methods (without using IdentitySourceProvider) which implement the converter interface as mentioned in the sample code provided here:
[documentation:]
https://github.com/SAP/cloud-security-xsuaa-integration/tree/main/spring-security#setup-spring-security-oauth-20-resource-server
sample link:
https://github.com/SAP/cloud-security-xsuaa-integration/blob/main/samples/spring-security-hybrid-usage/src/main/java/sample/spring/security/SecurityConfiguration.java
Error is :
Field authConverter in com.sap.acrs.config.WebSecurityConfig required a bean of type 'org.springframework.core.convert.converter.Converter' that could not be found.
The injection point has the following annotations:

@org.springframework.beans.factory.annotation.Autowired(required=true)

Library used: org.springframework.boot
spring-boot-starter-security
com.sap.cloud.security
resourceserver-security-spring-boot-starter
2.13.5
org.apache.httpcomponents httpclient - Environment: Cloud Foundry
Any kind of help would be appreciated.
Attaching my WebSecurityConfiguration file for reference.
WebSecurityConfiguration.zip

Hi @sharma-jahnvi ,
The issue is that the configuration that contains property xsappname could not be found in your environment.
For local testing, you can supply the configuration via application.yaml as described under this link:
https://github.com/SAP/cloud-security-xsuaa-integration/blob/main/spring-security/README.md#minimal-configuration-required
I hope that gets you going!
Best,
Manuel

Hi @finkmanAtSap
Hi Manuel
I had tried this already and also added the same in application .properties file to check if it works. But the issue still persists.
Actually the converter is not having any implementation class defined for it due to which the Spring context is not able to autowire the bean for it. Do we need to create a separate implementation class for this or is there any other way out ?
KIndly let me know in case of some solutions which can be worked upon.
Thanks
Jahnvi

Hey Jahnvi,
Sorry to hear that it is not working for you.
The Converter bean that should be found by default for Autowiring is defined in the following Configuration:
https://github.com/SAP/cloud-security-xsuaa-integration/blob/main/spring-security/src/main/java/com/sap/cloud/security/spring/autoconfig/HybridAuthorizationAutoConfiguration.java#L43
As you can see from the Conditional annotations in that file, the Converter bean would not be created if Spring property "sap.security.services.xsuaa.xsappname" is missing, hence my original suggestion to add it to the application.yaml or application.properties when running locally.
Also, the Configuration would not run at all if "sap.spring.security.hybrid.auto" is set to false.
As next step, can you please debug and check if this code section is executing for you?
And if not, double-check that the required properties are available?


[Feature request] Add edit button

The edit button was removed during the last update. Now, it is not possible to edit a registry without deleting it and creating it again. It would be better if the edit button came back with the new interface.

Solved by #9.


Replicate Rust Connector's Packet streaming via MessageBroker

Bias towards GCP PubSub to BigQuery, but consider Postgres as a sink.
Consider something like a correlation-id for aggregating packets into a single payment or per-account or some aggregator attribute.

Consider hooking-into prometheus via a channel.
Should the Pubsub publisher be called with .sync() or .async()? If async, the publish might fail, but the packet will propagate to the next ILP peer. This is fine for a very small packet amounts, but less-fine for large packet amounts.
Consider a flag for sync/vs async -- if sync, we can slow-down the pipeline, if async we might fulfill packets that don't make it to PubSub.
Since we can publish the same packets to PubSub, we need a GUID to allow BigQuery to distinguish published packets (because BigQuery can't distinguish two legit but identical packets otherwise).
Look at Rust impl to discover PubSub data format.

Implemented via #446


refactor: clean up error codes

Closes: #48
Signed-off-by: Jarkko Sakkinen<EMAIL_ADDRESS>

@haraldh I think this no brainer step but over time would be better to move address/length format for input. In other words: Address<usize, Page> and Offset<usize, Page>.


Rancher 2.2.2 HA cannot View/Edit Yaml when the yaml less than 2048 bytes

What kind of request is this (question/bug/enhancement/feature request):
BUG
Steps to reproduce (least amount of steps as possible):
any yaml less than 2048 bytes, use View/Edit yaml menu will get '[object Objec]' error.
In firefox, the request finished and response returns the base64 encoded yaml, but UI displays '[object Object].
In chrome, the request return error, and UI displays  '[object Object].
Environment information

Rancher version (rancher/rancher 2.2.2):
Installation option (single install/HA):
HA

Cluster information

Cluster type (Custom):
Machine type (VM) and specifications (32C/64G):
Kubernetes version (use kubectl version): 1.13.5
Docker version (use docker version): 18.9.5

Duplicate of https://github.com/rancher/rancher/issues/19723


Help needed with relationMappings and eager loading

How to reproduce
Very simplified tables
CREATE TABLE ideas (
    id BIGSERIAL NOT NULL primary key,
    text VARCHAR(1024)
);

CREATE TABLE cmt (
    id BIGSERIAL NOT NULL primary key,
    text VARCHAR(1024),
    idea BIGINT NOT NULL
)

Models
import { Model } from 'objection';

export default class Ideas extends Model {
  readonly id!: number;
  static tableName = 'ideas';
}

import { Model, RelationMappings } from 'objection';

export default class Cmt extends Model {
  static tableName = 'cmt';

  static modelPaths = [__dirname];

  static relationMappings: RelationMappings = {
    ideas: {
      relation: Model.BelongsToOneRelation,
      modelClass: 'ideas',
      join: {
        from: 'cmt.idea',
        to: 'ideas.id'
      }
    },
  };
}

A request somewhere
const withIdeas = await Cmt.query()
    .eager('ideas');

Response is:
Error: Cmt.relationMappings.ideas: join.from must have format TableName.columnName. For example "SomeTable.id" or in case of composite key ["SomeTable.a", "SomeTable.b"].

    at BelongsToOneRelation.createError (/home/vladimir/Projects/ucom/ucom.backend/node_modules/objection/lib/relations/Relation.js:232:14)
    at Object.createError (/home/vladimir/Projects/ucom/ucom.backend/node_modules/objection/lib/relations/Relation.js:46:32)
    at createRelationProperty (/home/vladimir/Projects/ucom/ucom.backend/node_modules/objection/lib/relations/Relation.js:346:17)
    at createJoinProperties (/home/vladimir/Projects/ucom/ucom.backend/node_modules/objection/lib/relations/Relation.js:302:20)
    at BelongsToOneRelation.setMapping (/home/vladimir/Projects/ucom/ucom.backend/node_modules/objection/lib/relations/Relation.js:54:11)
    at Object.keys.reduce (/home/vladimir/Projects/ucom/ucom.backend/node_modules/objection/lib/model/Model.js:735:29)
    at Array.reduce (<anonymous>)
    at getRelations (/home/vladimir/Projects/ucom/ucom.backend/node_modules/objection/lib/model/Model.js:731:46)
    at cachedGet (/home/vladimir/Projects/ucom/ucom.backend/node_modules/objection/lib/model/Model.js:679:61)
    at Function.getRelations (/home/vladimir/Projects/ucom/ucom.backend/node_modules/objection/lib/model/Model.js:465:12)
    at WhereInEagerOperation.onAdd (/home/vladimir/Projects/ucom/ucom.backend/node_modules/objection/lib/queryBuilder/operations/eager/WhereInEagerOperation.js:36:34)
    at QueryBuilder.addOperation (/home/vladimir/Projects/ucom/ucom.backend/node_modules/objection/lib/queryBuilder/QueryBuilderOperationSupport.js:234:33)
    at addEagerFetchOperation (/home/vladimir/Projects/ucom/ucom.backend/node_modules/objection/lib/queryBuilder/QueryBuilder.js:1259:13)
    at beforeExecute (/home/vladimir/Projects/ucom/ucom.backend/node_modules/objection/lib/queryBuilder/QueryBuilder.js:1138:5)
    at Bluebird.try (/home/vladimir/Projects/ucom/ucom.backend/node_modules/objection/lib/queryBuilder/QueryBuilder.js:568:31)
    at tryCatcher (/home/vladimir/Projects/ucom/ucom.backend/node_modules/bluebird/js/release/util.js:16:23)
    at Function.Promise.attempt.Promise.try (/home/vladimir/Projects/ucom/ucom.backend/node_modules/bluebird/js/release/method.js:39:29)
    at QueryBuilder.execute (/home/vladimir/Projects/ucom/ucom.backend/node_modules/objection/lib/queryBuilder/QueryBuilder.js:568:24)
    at QueryBuilder.then (/home/vladimir/Projects/ucom/ucom.backend/node_modules/objection/lib/queryBuilder/QueryBuilder.js:473:26)
    at process._tickCallback (internal/process/next_tick.js:68:7)

The error is happened here - empty array is returned
https://github.com/Vincit/objection.js/blob/master/lib/relations/RelationProperty.js#L205
A dirty fix:
https://github.com/Vincit/objection.js/blob/master/lib/utils/objectUtils.js#L64
change the last line
from
return Array.from(map.values());
to
return Array.from(map);

Please, help. What I'm doing wrong? :(

Yes it does.

Oh, there was a question at the bottom. I thought this was just one of those "your code that has 3600 tests doesn't work!#"!#!"#!"!!" issues.
Is the name of the ideas model file ideas.js (after compilation to js)?

Thank you a lot for the quick response, and I'm sorry about not to be obvious enough.
After searching a lot, I finally found a 3rd party dependency, which breaks eager relation. It is not a problem related to PostgreSQL.
Please observe a test project to investigate it.
https://github.com/vladimirice/objection-postgresql
If I add this dependency, an error occurs.
https://github.com/vladimirice/objection-postgresql/blob/master/example.js#L8
I use this dependency for one of my projects. A project above is just a sample project to reproduce an issue.
Maybe It is not a problem of objectionjs. Maybe a dependency 'node-dependency-injection' breaks something or use some dirty tricks. But maybe not. I'll be pleased if you can take a look at what is happening.
Maybe this issue can help you to improve something in objection.js. Thanks!

The conflict happens when require('collections')
https://github.com/montagejs/collections
Nothing related to node-dependency-injection

Is it possible to somehow avoid objectionjs errors related to collections usage?

It's possible, but I won't do that. No library should ever change the built-in methods like Array.find. You should drop whatever library that does this or put your efforts towards those libraries dropping these evil practices. It's insanely dangerous (and quite frankly stupid) to override the built-ins.


Update BaseViewController

With commit https://github.com/naoyashiga/Dunk/commit/af99461c453f990ce0fac715b8d39cb6392f9757, we no longer need to set the parentNavigationController for ShotCollectionViewController in BaseViewController. This PR addresses a compile issue with the last round of changes.

thanks! :+1:


Clarification question about semantic entropy computation

Hey, thanks a lot for publishing the code!
I was hoping you could shed some light on the following two lines.
First, I don't immediately see why torch.max(torch.nan_to_num(torch.logsumexp(mean_across_models, dim=1), nan=-100)) is subtracted from mean_across_models
https://github.com/lorenzkuhn/semantic_uncertainty/blob/2cbd5a5d4ac5386ebe98526b302e5fda3d4f5a65/code/compute_confidence_measure.py#L126
Second, what does llh_shift represent, and why it is subtracted from torch.tensor(aggregated_likelihoods)?
https://github.com/lorenzkuhn/semantic_uncertainty/blob/2cbd5a5d4ac5386ebe98526b302e5fda3d4f5a65/code/compute_confidence_measure.py#L136
Let me know if you want me to be more specific.
Thanks again!
Joris

Hi Joris,
Thank you for your interest in our work!
In early experiments, we used the first line you mention to address numerical issues. This was not used in our final experiments and was accidentally included in our release version. I've updated the released version of the code to reflect that.
The llh_shift is a heuristic way of accounting for the fact that we're summing up unnormalized likelihoods when computing the semantic entropy. We are summing up unnormalized probabilities when using length-normalized sequence likelihoods. By applying the llh_shift we ensure that all the semantic log-likelihoods are negative before computing the Monte Carlo estimate of the entropy over them.
I hope this helps!
Lorenz

Got it, thank you


[API] Improve message when a peer is banned.

Fixes 3105

Also, did you check that in bitcoin, addnode sent with a banned node will  not remove the ban?

Merge with master and check for failing tests.

I think this is not the correct way to implement this.
There is an endpoint now to know if ht epeer is banned so that endpoint can be called.

Closing as will re-implement


Dialog focus management fix

Return focus to previously active element on dialog close.

Most noticeable on "Start" button on home page and record button on the add data page.


In tests, do not connect to unspecified addrs

Windows doesn't seem to handle it

r? @vinipsmaker
(maidsafe_highfive has picked a reviewer for you, use r? to override)


import uikit components

Hi,
How can i import Uikit 2 components without cdn links ?

Hi @miladmeidanshahi
I don't know for UIkit 2, but with UIkit 3 it works fine.
See uikit example and live demo.
Glitch: https://glitch.com/edit/#!/nuxt-uitkit


Wrong debug info

Sometimes I'm seeing stacktraces with frames having wrong line (and possible column) numbers.
It seems to me there's some overflow going on (0 / 255 values).
Example stacktrace - notice column numbers with 0 and 255:
Can only invoke 'to_s' once on String::Builder (Exception)
  from /usr/local/Cellar/crystal/0.27.2/src/string/builder.cr:0:5 in 'to_s'
  from src/raven/processors/sanitize_data.cr:28:5 in 'sanitize_query_string'
  from src/raven/processors/sanitize_data.cr:68:11 in 'process'
  from src/raven/processors/sanitize_data.cr:45:21 in 'process'
  from src/raven/processors/sanitize_data.cr:56:9 in 'process'
  from src/raven/processors/sanitize_data.cr:45:21 in 'process'
  from src/raven/processors/sanitize_data.cr:56:9 in 'process'
  from src/raven/processors/sanitize_data.cr:45:21 in 'process'
  from spec/raven/processors/sanitize_data_spec.cr:259:7 in '->'
  from /usr/local/Cellar/crystal/0.27.2/src/spec/methods.cr:255:3 in 'it'
  from spec/raven/processors/sanitize_data_spec.cr:249:5 in '->'
  from /usr/local/Cellar/crystal/0.27.2/src/spec/context.cr:158:7 in 'describe'
  from /usr/local/Cellar/crystal/0.27.2/src/spec/methods.cr:16:5 in 'describe'
  from /usr/local/Cellar/crystal/0.27.2/src/spec/methods.cr:25:5 in 'context'
  from spec/raven/processors/sanitize_data_spec.cr:200:3 in '->'
  from /usr/local/Cellar/crystal/0.27.2/src/spec/context.cr:255:3 in 'describe'
  from /usr/local/Cellar/crystal/0.27.2/src/spec/methods.cr:16:5 in 'describe'
  from spec/raven/processors/remove_stacktrace_spec.cr:1:1 in '__crystal_main'
  from /usr/local/Cellar/crystal/0.27.2/src/crystal/main.cr:97:5 in 'main_user_code'
  from /usr/local/Cellar/crystal/0.27.2/src/crystal/main.cr:86:7 in 'main'
  from /usr/local/Cellar/crystal/0.27.2/src/crystal/main.cr:106:3 in 'main'

Tested with Crystal 0.27.2 and macOS 10.13.6.

still present? how to reproduce?

Can you check in gdb/lldb (add a breakpoint before the raise and then print the backtrack when execution stops) and report whether the line number is correct or zero?
First case means there is a bug in the dwarf decoder, zero means there is a bug in the debug information passed to llvm ir.

Is there a reproducible example?

I've reproduced it with kemal, perhaps there are other prerequisites.

It's quite easy to reproduce. I spot it from time to time but never gathered the courage to analyze it.

It might have sth to do with @[AlwaysInline] annotation 🤔

I just ran into this (linux, OS X is OK) see top line of this backtrace:
https://github.com/crystal-lang/crystal/issues/7383#issuecomment-550862925

OK I was asked to add some other "classes" of wrong debug info to this issue, so I guess it'll be a grab bag of sorts...
Here's one:
Here's how to repro.  Linux ubuntu.  From a crystal git repo:
diff --git a/src/socket.cr b/src/socket.cr
index fb25d63ae..3da6719cf 100644
--- a/src/socket.cr
+++ b/src/socket.cr
@@ -544,6 +544,7 @@ class Socket < IO
   end

   private def unbuffered_write(slice : Bytes)
+    raise "hello"
     evented_write(slice, "Error writing to socket") do |slice|
       LibC.send(@fd, slice, slice.size, 0)
     end

Use it somehow:
crystal$ ./bin/crystal  spec/std/openssl/ssl/socket_spec.cr
Unhandled exception in spawn: hello (Exception)
  from src/socket.cr:547:5 in 'unbuffered_write'
  from src/io/buffered.cr:135:14 in 'write'
  from src/openssl/bio.cr:31:7 in '->'
  from src/slice/sort.cr:147:5 in 'heapify!'
  from src/slice/sort.cr:153:10 in 'center_median!'
  from src/slice/sort.cr:204:0 in 'cmp'
  from src/slice/sort.cr:156:9 in 'center_median!'
  from ???
  from ???
  from ???
  from SSL_do_handshake
  from src/openssl/ssl/socket.cr:32:15 in 'initialize'
  from src/openssl/ssl/socket.cr:3:5 in 'new'
  from spec/std/openssl/ssl/socket_spec.cr:43:7 in '->'
  from src/fiber.cr:255:3 in 'run'
  from src/fiber.cr:47:34 in '->'
  from ???

Same trace, in gdb
$ gdb socket_spec
(gdb) b src/socket.cr:547
(gdb) r
(gdb) bt
#0  unbuffered_write () at /home/joshua/dev/crystal/src/socket.cr:547
#1  0x0000555555682c63 in write () at /home/joshua/dev/crystal/src/io/buffered.cr:135
#2  0x00005555555d5bb5 in -> () at /home/joshua/dev/crystal/src/openssl/bio.cr:31
#3  0x00007ffff7cfea33 in ?? () from /lib/x86_64-linux-gnu/libcrypto.so.1.1
#4  0x00007ffff7cfeed3 in BIO_write () from /lib/x86_64-linux-gnu/libcrypto.so.1.1
#5  0x00007ffff7cfd02a in ?? () from /lib/x86_64-linux-gnu/libcrypto.so.1.1
#6  0x00007ffff7cff3a7 in BIO_ctrl () from /lib/x86_64-linux-gnu/libcrypto.so.1.1
#7  0x00007ffff7f6d3bd in ?? () from /lib/x86_64-linux-gnu/libssl.so.1.1
#8  0x00007ffff7f71e0f in ?? () from /lib/x86_64-linux-gnu/libssl.so.1.1
#9  0x00007ffff7f6c89a in ?? () from /lib/x86_64-linux-gnu/libssl.so.1.1
#10 0x00007ffff7f589c4 in SSL_do_handshake () from /lib/x86_64-linux-gnu/libssl.so.1.1
#11 0x00005555556b0890 in initialize () at /home/joshua/dev/crystal/src/openssl/ssl/socket.cr:32
#12 0x00005555556b06c9 in new () at /home/joshua/dev/crystal/src/openssl/ssl/socket.cr:3
#13 0x00005555555d78ad in -> () at /home/joshua/dev/crystal/spec/std/openssl/ssl/socket_spec.cr:43
#14 0x0000555555614123 in run () at /home/joshua/dev/crystal/src/fiber.cr:255
#15 0x00005555555d2c96 in -> () at /home/joshua/dev/crystal/src/fiber.cr:47
#16 0x0000000000000000 in ?? ()

Appears crystal is identifying some lines as part of the crystal std lib crystal files (heapify et al lines) that are actually part of a 3rd party library.

Similarly, from #8534 those from ??? lines could show the filename of the 3rd party library they come from.

Also somewhat related: #8352

Possibly related? the question marks in the backtrace here: https://github.com/crystal-lang/crystal/issues/8584

#9209 may have some clues

I'm hoping this was fixed in 2d02e52d9d7438bbcadcf222eed0fce2a7ca56d0


Add barrier instructions validation pass

Fails Vulkan CTS master.

Manually tested on on VS2017 Release.  Passes.

Rebased, squashed and pushed into master as 12e6860d073b


Implement dynamic ranges

Closes #4021
Allows array ranges (e.g., [0..10]) to take expression instead of just numeric literals as their start and end values. Both expressions are required (we don't support [0..], etc.).
I've created a new kind of expression in the AST. The alternative was to represent the internals of an array as some kind of pattern which could initially be fully explicit or ranges. I figured the chosen version was simpler and easier to extend to open ranges, whereas the latter would be easier to extend to mixed ranges or other patterns. I chose simpler, it'll be easy enough to refactor if necessary.
Parsing is tested implicitly by the tests of execution and unparsing.

I'm surprised this didn't need TypeScript changes to pass TS type checking and tests.

Me too! As far as I could tell, on the TS side it was expecting expressions rather than literals already, so there was nothing to change. I could easily have missed something though

Thanks @nrc!


AttributeError: frame_options

Hello,
We have a Flask app with Talisman and we initialize the app by default values:
csp = {
        'default-src': '\'self\'',
        'img-src': '\'self\' data:',
        'media-src': [
            '*',
        ],
        'style-src': '\'unsafe-inline\' \'self\'',
        'script-src': '\'unsafe-inline\' \'self\'',
        'font-src' : '*'
    }
    Talisman(app, content_security_policy=csp)

But sometimes, we are not sure why, it's hard to reproduce we have the following error and stacktrace :asd
Traceback (most recent call last):
  File "/root/19032018/asd/venv/lib/python3.6/site-packages/flask/app.py", line 2000, in __call__
    return self.wsgi_app(environ, start_response)
  File "/root/19032018/asd/venv/lib/python3.6/site-packages/flask/app.py", line 1991, in wsgi_app
    response = self.make_response(self.handle_exception(e))
  File "/root/19032018/asd/venv/lib/python3.6/site-packages/flask/app.py", line 1567, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "/root/19032018/asd/venv/lib/python3.6/site-packages/flask/_compat.py", line 33, in reraise
    raise value
  File "/root/19032018/asd/venv/lib/python3.6/site-packages/flask/app.py", line 1988, in wsgi_app
    response = self.full_dispatch_request()
  File "/root/19032018/asd/venv/lib/python3.6/site-packages/flask/app.py", line 1643, in full_dispatch_request
    response = self.process_response(response)
  File "/root/19032018/asd/venv/lib/python3.6/site-packages/flask/app.py", line 1862, in process_response
    response = handler(response)
  File "/root/19032018/asd/venv/lib/python3.6/site-packages/flask_talisman/talisman.py", line 210, in _set_response_headers
    self._set_frame_options_headers(response.headers)
  File "/root/19032018/asd/venv/lib/python3.6/site-packages/flask_talisman/talisman.py", line 217, in _set_frame_options_headers
    headers['X-Frame-Options'] = self.local_options.frame_options
  File "/root/19032018/asd/venv/lib/python3.6/site-packages/werkzeug/local.py", line 72, in __getattr__
    raise AttributeError(name)
AttributeError: frame_options

Can you help why this happens and why it happens at seemingly random times?
Talisman version is 0.4.1
Thanks in advance!

That's pretty wild. What is your wsgi server setup?

We do not use wsgi server on our application as far as i know. Here is our run code
from app import app
import meinheld

app = app.create_app()

if _name_ == '_main_':
    if app.config['PROD']:
        meinheld.listen(("<IP_ADDRESS>", 5000))
        meinheld.run(app)
    else:
        app.run(host='<IP_ADDRESS>', port=5001, threaded=True, debug=True)

It seems meinheld is your wsgi sever which uses greenlet. I'm not sure if this is a race condition or not due to using greenlet, but that's my first suspicion.

Gevent==1.1.2
PyMongo==3.4.0
meinheld==0.6.0
mongoengine==0.10.7
gunicorn==19.6.0
marshmallow-mongoengine==0.7.7
python-slugify==1.2.1
Flask==0.11.1
Flask-WTF==0.13.1
Flask-MongoEngine==0.8
Flask-DebugToolbar==0.10.0
Flask-Security==1.7.5
Flask-Mail==0.9.1
Flask-Script==2.0.5
boto==2.38.0
boto3==1.4.8
Flask-Babel==0.11.2
confluent-kafka==0.11.0
redis==2.10.6
zeep==2.5.0
Flask-Excel==0.0.7
pyexcel-xlsx==0.5.5
uvloop
aiokafka
apscheduler
captcha==0.2.4
flask-session-captcha==1.1.0
flask-sessionstore==0.4.5
flask_sqlalchemy==2.3.2
flask-login==0.3.2
ldap3==2.4.1
pyldap
flask-simpleldap
flask-talisman==0.4.1

These are our requirements for the project. Is this a library version problem? Should we upgrade/downgrade anything flask related?

I think it's a race condition / non-thread safe issue. Not 100% sure why it's occurring though.

Same happened here with gunicorn
Python 3.6.4
Flask 0.12.2
Gunicorn 19.7.1
# coding: utf-8
'Gunicorn configuration'

import os

# See http://docs.gunicorn.org/en/latest/settings.html
threads = 1
workers = os.environ.get('WEB_CONCURRENCY', 2)
reload = os.environ['ENV'] == 'development'
preload_app = True

Also tried without app preloading just to try.
FYI it doesn't happen when Flask is in debug mode using flask run, it does not happen even with gunicorn with flask in debug mode, it only happens if you run gunicorn locally with flask in "production" mode.

Thanks, @rhymes that certainly seems like a race condition. Pull requests welcome, as I'm unsure when I'll be able to personally take a look at this.

fixed this temporarily by adding these 2 lines in talisman.py

policy = self.local_options.content_security_policy

by;
try:
        policy = self.local_options.content_security_policy
    except Exception as e:
        policy  = {
                    'default-src': '\'self\'',
                    'img-src': '\'self\' data:',
                    'font-src' : '*'
                    }

if not self.local_options.content_security_policy:

by;
    try:
        if not self.local_options.content_security_policy:
            return
    except Exception as e:
        pass

hope this helps for someone till @jonparrott will solve the issue.
regards,

@davidism would you happen to have any ideas why this might be occurring? Talisman sets local frame options in before_request to a werkzeug Local object and then uses them in after_request. It seems like some sort of race condition or concurrency issue, but I can quite figure out why.

I've been following this, I'll try to look at it when I have time, maybe this weekend.

I'm not able to reproduce this. I've tried making a ton of concurrent requests to the dev server, Gunicorn, and Meinheld with no errors. Is there a full example I can use that reproduces this?

@davidism I'm able to reproduce it in my local environment, just running the local server with "flask run".
The strange thing is that If I visit the main route "http://localhost:5000" I got the error.
Then if I surf to "http://localhost:5000/en" or anything as param "http://localhost:5000/foo" the error disappear and everything works fine.
Here my pip list:
alembic (0.8.10)
appdirs (1.4.0)
appnope (0.1.0)
APScheduler (3.3.1)
asn1crypto (0.24.0)
Babel (2.5.3)
backports.shutil-get-terminal-size (1.0.0)
bcrypt (3.1.4)
blinker (1.4)
boto3 (1.4.4)
botocore (1.5.8)
cffi (1.11.5)
click (6.7)
cryptography (2.2.2)
cssmin (0.2.0)
decorator (4.0.11)
docutils (0.13.1)
enum34 (1.1.6)
facebook-sdk (2.0.0)
Flask (0.12)
Flask-APScheduler (1.6.0)
Flask-Babel (0.11.2)
Flask-BabelEx (0.9.3)
Flask-Login (0.4.1)
Flask-Mail (0.9.1)
Flask-Principal (0.4.0)
Flask-Security (3.0.0)
Flask-Session (0.3.1)
flask-talisman (0.5.0)
Flask-Webpack (0.1.0)
Flask-WTF (0.14.2)
funcsigs (1.0.2)
futures (3.0.5)
gyp (0.1)
idna (2.6)
ipaddress (1.0.19)
ipdb (0.11)
ipython (5.6.0)
ipython-genutils (0.2.0)
itsdangerous (0.24)
Jinja2 (2.9.5)
jmespath (0.9.1)
jsmin (2.2.2)
Mako (1.0.6)
MarkupSafe (0.23)
MySQL-python (1.2.5)
netaddr (0.7.19)
olefile (0.44)
packaging (16.8)
passlib (1.7.1)
pathlib2 (2.2.1)
paypalrestsdk (1.13.1)
pexpect (4.2.1)
phonenumbers (8.4.3)
pickleshare (0.7.4)
Pillow (4.0.0)
pip (8.1.1)
prompt-toolkit (1.0.15)
ptyprocess (0.5.2)
pycparser (2.18)
Pygments (2.2.0)
pyOpenSSL (18.0.0)
pyparsing (2.2.0)
python-dateutil (2.6.0)
python-editor (1.0.3)
python-magic (0.4.12)
pytz (2018.4)
querystring-parser (1.2.3)
redis (2.10.5)
redis-py-cluster (1.3.4)
requests (2.13.0)
rivescript (1.14.4)
s3transfer (0.1.10)
scandir (1.4)
setuptools (39.0.1)
simplegeneric (0.8.1)
six (1.11.0)
speaklater (1.3)
SQLAlchemy (1.1.6)
stripe (1.51.0)
traitlets (4.3.1)
tzlocal (1.3)
virtualenv (15.0.1)
wcwidth (0.1.7)
webassets (0.12.1)
Werkzeug (0.11.15)
wheel (0.29.0)
WTForms (2.1)

Unfortunately, that list won't help me reproduce this. Starting from scratch, what is the smallest program and set of steps I need to take to see this?

for me to reproduce, i just wrote the sample code and talisman is ALWAYS giving the frame_options error. maybe it'll help you.
in app.py
__all__ = ['create_app']

def create_app(config=None, app_name=None, blueprints=None):

    app = Flask(app_name)
    configure_app(app, config)
    configure_hook(app)

def configure_hook(app):
    @app.before_request
    def set_unique_token():
        token = session.get('unique_token', None)
        if token is None:
            session['unique_token'] = uuid1()
            if not current_user.is_authenticated:
                return redirect("/")

when I deleted the line
if not current_user.is_authenticated:
                return redirect("/")

it is hard to reproduce the error but when I wrote that 2 lines it is ALWAYS giving the error and the UI is not working. by reproducing the error with these 2 lines, I wrote my config in talisman.py. It does not give me any error by now.

I am running into this bug in my own code base. I trimmed down my codebase to a fairly minimal example that always produces this error.
The code is in a file called server.py
from flask import Flask, Response
from flask_talisman import Talisman

app = Flask(__name__)

class CustomException(Exception):
    pass

@app.before_request
def throw_exception():
    raise CustomException()

talisman = Talisman(app)

@app.route('/route')
def route():
    return {}

@app.errorhandler(CustomException)
def csrf_error(exception):
    return Response(status=400)

app.run(debug=True)

I run this with the command python3 server.py.
I get the error when I run the following command: curl <IP_ADDRESS>:5000/route
It is worth noting that moving the line talisman = Talisman(app) above the line @app.before_request causes the bug to no longer reproduce.

I believe the issue is that an exception being thrown in an @app.before_requestmethod will cause all remaining @app.before_request methods not to execute. This can be seen in the way Flask handles this functions: https://github.com/pallets/flask/blob/1fa9185c7e82ecc43b99d797af5584133458008d/flask/app.py#L2064-L2089
In Flask-Talisman, we use @app.before_request and @app.after_request here: https://github.com/GoogleCloudPlatform/flask-talisman/blob/master/flask_talisman/talisman.py#L165-L168
Since _update_local_options is not getting run after the exception is thrown, the local_options object is not having its attributes set. When we try to access frame_options in the @app.after_request method, it throws.

I am not sure what the correct solution is to this issue. Here are some ideas:

We consider this a bug in Flask and update Flask to continue running @app.before_request methods even if one of them throws.
We consider this a bug in Flask-Talisman and try to modify it to not count on @app.before_request running.

I will open an issue on Flask's repo to discuss whether or not Flask should continue running @app.before_request methods after an exception is thrown.

As a hacky workaround, I am adding Flask-Talisman before calling @app.before_request so that Flask-Talisman runs first.

This needs to be fixed here, not in Flask.

Also, thanks to both the examples, it really helped narrow this down.

@skylerberg this did not work for me or could you please write down your piece of code here so that I can see what I am missing? thanx

Thanks, everyone for figuring out why this is occurring. I'm going to try to fix today. :)

I've upload 0.5.1 to PyPI which (hopefully) fixes this. Thanks everyone for helping me figure it out, and apologies that it took so long to fix!


Office 365-Based Adaptive Authentication Script Not Functioning Properly

Describe the issue:
Office365-based Adaptive Authentication default script is not working since it has the domain in the roleToBeAssigned variable.
Note: Once the Internal  part is removed the script works as expected.
How to reproduce:

Create a role named office365Role
Go to the selected app and from sign-in method tab enable the Update office365 role script
Assign the created role to the app from the Roles tab
try the sign-in flow
Find that the defined role is not assigned to the logged-in user.

Expected behavior:

Environment information (Please complete the following information; remove  any unnecessary fields) :

Product Version: [e.g., IS 5.10.0, IS 5.9.0]
OS: Mac
Database: MySQL
Userstore: [e.g., LDAP, JDBC]

Optional Fields
Related issues:

Suggested labels:

As discussed with @AnuradhaSK let's modify this function hasAnyOfTheRolesV2 to ignore the domain and validate the role. For example in this case the role needs to be assigned in both cases when the role is Internal/office365Role or office365Role

This issue is fixed from following PR.

https://github.com/wso2-extensions/identity-conditional-auth-functions/pull/153


Create Cake build script

Fixes #123 creating a cake script to do build, test and packaging for the adapter.

@OsirisTerje For your review

Small comment on the title, to include VS2015 - or just say Visual Studio.  We do have a description where we could state more details, like >= VS2012.  Otherwise looks good.


julius: darwin build, add maintainer

Description of changes
Added Darwin support to Julius.
List of changes:

Disabled fixup_bundle CMake directive
Disabled unit tests
Exposed symlinks to darwin utils sips, iconutil and codesign (is this allowed, or is there a better way to do this?)

Those 3 binaries have been around (together) starting from OS X Lion

Added myself as second maintainer for Darwin

The screenshot below is from the compiled julius.app running on my M1 machine (macOS 14.5).

Binaries tested both on aarch64-darwin and aarch64-linux.
Things done

Built on platform(s)

[ ] x86_64-linux
[X] aarch64-linux
[ ] x86_64-darwin
[X] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[X] Tested basic functionality of all binary files (usually in ./result/bin/)
24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[X] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/prs-ready-for-review/3032/4049

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/prs-ready-for-review/3032/4058

@paparodeo @SuperSandro2000 ready for re-review, thanks for the feedback so far!

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/prs-already-reviewed/2617/1733

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/prs-already-reviewed/2617/1737

@SuperSandro2000 friendly ping to ask for merge, thank you 🙏🏻


Adds EditKit Pro by @aryamansharda

[X] Project URL: https://apps.apple.com/us/app/editkit-pro/id1659984546
[X] Update contents.json instead of README
[X] One project per pull request
[X] Screenshot included
[X] Avoid iOS or open-source in description as it is assumed
[X] Use this commit title format if applicable: Add app-name by @github-username
[X] Use approved format for your entry

Note: The link in the screenshots section includes demo .gifs for each piece of functionality individually.

Sorry this list is for iOS apps
Closing


Update XpandNavigationItem.cs

Changed "XpandNavigationItemAttribute" to add caption.
Changed "XpandNavigationItem2NodeUpdater" to

Make it simpler to read
Implement the caption change

Comments removed

I changed the "XpandNavigationItemAttribute" to add caption property.

Then changed "XpandNavigationItem2NodeUpdater"
removed method "AddNodes" which is recursive and complex  with simple one with simple foreach method "GetParentNavigationItems"
also removed method "GetNavigationItem"  with method "AddNavigationItem"

I suggest you to not compare the code, Rather read my code alone to see the new idea

for future reference the new idea should be the commit message and not the refactoring u did. For example XpandNavigationItemAttribute support for Caption
I will cleanup this request manually for now but I will retain your authorship. Thanks for the time sending the request, feel free to send more. see  also https://github.com/eXpandFramework/eXpand/issues/260

Installer lab build 18.2.301.21 includes commit "XpandNavigationItemAttribute support for Caption #260 #258" that relate to this task. Please test if it addresses the problem. If you use nuget add our NugetServer as a nuget package source in VS

Installer lab build 18.2.301.67 includes commit "XpandNavigationItemAttribute support for Caption #260 #258" that relate to this task. Please test if it addresses the problem. If you use nuget add our NugetServer as a nuget package source in VS

include in 18.2.302