monaco.languages pointing to a different object in library and consumer

Hi 😄
I'd like to provide completely custom language, and wrap this custom language into a separate library.
But when I do that, the completion logic doesn't work.
What works
If the completion logic registration is within the main project, everything works as expected:

Here is how the main project looks like:
init.ts (dummy completion item provider)
const provideCompletionItems = (model: monaco.editor.ITextModel, position: monaco.Position): monaco.languages.ProviderResult<monaco.languages.CompletionList> => {
    var textUntilPosition = model.getValueInRange({startLineNumber: 1, startColumn: 1, endLineNumber: position.lineNumber, endColumn: position.column});
    var word = model.getWordUntilPosition(position);
    var range = {
        startLineNumber: position.lineNumber,
        endLineNumber: position.lineNumber,
        startColumn: word.startColumn,
        endColumn: word.endColumn
    };

    return {
        suggestions: createDependencyProposals(range)
    };
}

export const init = (languageId: string) => {
    monaco.languages.register({
        id: languageId
    });

    monaco.languages.registerCompletionItemProvider('languageId', {
        provideCompletionItems: provideCompletionItems
    });
}

MyEditor.tsx (React component)
import React from "react";
import ReactMonacoEditor from "react-monaco-editor";
import * as monacoEditor from "monaco-editor/esm/vs/editor/editor.api";
import { init } from "./init"; // <-- importing from the same project

interface Props {
    value?: string
}

const DefaultLanguage = 'foo'

class MyEditor extends React.Component<Props> {
  editorWillMount(monaco: typeof monacoEditor) {
      init(DefaultLanguage); // <-- initializing the completion
  }

  render() {
    return (
      <ReactMonacoEditor
        language={DefaultLanguage}
        editorWillMount={this.editorWillMount}
        value={this.props.value}
      />
    );
  }
}

export default MyEditor

What doesn't work
If I move the "init" code to another library, and import this library into the main project, the completion item list is empty:

import { init } from "./my-library"; // <-- import from another library

As said, the library exports the same "init" code as above (Typescript compiled to Javascript as UMD modules).
Here's an excerpt of webpack.config:
output: {
    libraryTarget: 'umd',
},

Works if passing a reference to monaco from the consumer
I discovered that if I pass the reference to monaco from the consumer, everything works as expected:

MyEditor.tsx  change
editorWillMount(monaco: typeof monacoEditor) {
    init(monaco, DefaultLanguage) // <-- passing monaco
}

my-library/init.ts
const provideCompletionItems = () => {} // same as above

export const init = (consumerMonaco: typeof monacoEditor, languageId: string) => { // <-- passing a reference as consumerMonaco
    monaco.languages.register({
        id: languageId
    });

    consumerMonaco.languages.registerCompletionItemProvider('languageId', { // <-- using consumerMonaco here
        provideCompletionItems: provideCompletionItems
    });

    console.log('monaco === callerMonaco', monaco === consumerMonaco);
}

Console.log prints out the comparison between the "inner" and "outer" monaco, and it turns out these two are not the same object:

I'm wondering what's the reason for this?
I think it's very counter-intuitive. It took me literally days to figure this out. 😕
monaco-editor version: 0.21.2
Browser: Chrome 87.0.4280.88
OS: Windows

@alexdima I bumped up the monaco-editor version to the latest (0.23.0) to get this change.
However, in my project the following features stopped working:

code coloring
auto-completion

I don't see any errors in the console.
I'd appreciate if you could help me realizing what changed to monaco-editor between versions 0.21.2 and 0.23.0 and how should I make everything work again.
As for the auto-completion, I am actually getting the "Monarch" (hardcoded) auto-completion items (provided by monaco.languages.setLanguageConfiguration()).
However, I'm not getting any items from completion provider (registered by monaco.languages.registerCompletionItemProvider()).
Here's my code:
const registerLanguage = (languageId: string, moduleId: string, urlTemplates: QueryEditorUrlTemplates ) => {
    console.log('registerLanguage', languageId);

    monaco.languages.register({
        id: languageId
    });

    monaco.languages.setLanguageConfiguration(languageId, queryLanguageConfiguration);

    monaco.languages.onLanguage(languageId, () => {
        console.log('onLanguage', languageId);

        // code coloring
        monaco.languages.setMonarchTokensProvider(languageId, getMonarchTokensProvider(languageId));

        // creating worker
        const worker = createWorker(moduleId, languageId, config);

        // intellisense
        monaco.languages.registerCompletionItemProvider(languageId, new CompletionItemProvider(worker));
    });
}

So, for the monaco editor with language "foo", I'm getting both these line printed out in the console:
registerLanguage foo
onLanguage foo

My completion provider:
export class CompletionItemProvider implements monaco.languages.CompletionItemProvider {

This provider implements triggerCharacters and provideCompletionItems (which is never called).
monaco-editor version: 0.23.0
Browser: Chrome 87.0.4280.88
OS: Windows


Select instrumentation tooling (assumes OpenTelemetry exporter for Azure Monitor)

As a project lead, I want to make a decision on what tooling to use to instrument telemetry data, so that telemetry data can be collected and sent to the centralized monitoring solution

Mentioned in #228


small test change that catches the CORE-3964 bug

Tested that 604e9ecd219f413cf03f3ba50f2ed0e91ef29013 fails with this test expansion, so good!   Login totally fails against the old code with this test.


istioctl x es adopt prometheus annotations

Please provide a description of this PR:
istioctl x es adopt prometheus annotations when output is prom-merged
this will make you can retrieve stats for ztunnel by running:
 out/linux_amd64/istioctl x es -nistio-system ztunnel-wbtp4 -o prom-merged

kindly ping @istio/wg-environments-maintainers


Wiki  incorrect example

Bug Report

🔎 Search Terms
I am reading about typescript and encountered incorrect example (in my opinion)
Example
🕗 Version & Regression Information
30.05.2023
🙁 Actual behavior
The method someCallback doesn't even called, and this.x prints 10, not undefined
🙂 Expected behavior
I think we can fix this example, or remove

I'm not sure what you're getting at. Try running this code in your browser console; it behaves as described


Conditional Access App Control "Block Downloads" does not block jpg download

Hello,
Conditional Access App Control "Block Downloads" does not block files with the ".jpg" extension download. I also tried to create a Session Control policy in MCAS to block jpg download but it does not work either.
You can block jpg upload but not download.

@Martony78 Thanks for contacting Microsoft about your issue. As we understand it, you're describing a problem with the product itself and not with the product documentation.
I'll check with the product team though it could take from a few business days to a few weeks to look further into your issue. We'll contact you if we need further information, and you'll receive an email message each time we "@mention" you in a comment.
To get assistance or support for your product issue, please open a support ticket.

Hello,
As I can see png files are also concerned... it seems like image files are the issue. The problem is if we rename any file to jpg or png we can download it from any browser.

@Martony78 I discussed the issue with the product team and since the problem is with the product itself and not with the product documentation, they requested that you email<EMAIL_ADDRESS>where you can get further assistance on this issue. In the meantime, I'm closing this issue. Thanks again for your feedback.


[WFCORE-4155] Upgrade to Galleon and WildFly Galleon Plugins 2.0.1.Final

https://issues.jboss.org/browse/WFCORE-4155

Core - Full Integration Build 8036 outcome was FAILURE using a merge of 0b8c4d9a6aa16ee409e34f75e4d0bed73f6bcee9
Summary: Tests passed: 385, ignored: 28; exit code 1 (Step: Build & test full (Maven)) (new) Build time: 00:03:56

The integration tests fail due to the Galleon and WFGP version conflict between WildFly Core and WildFly Full. There are two ways to proceed:

merge the corresponding PR for WildFly https://github.com/wildfly/wildfly/pull/11719;
include WildFly provisioning plugin into Servlet feature-pack and not inherit it from Core.

Either way it requires a change in WildFly Full build though.


Empty Rules no longer prevent a Policy from applying

Apparently if a Policy points to an empty/non-existent Rule, that Policy still applies (unless of course prevented by other Rules). This is now reflected in our code too. (Also, Policies that do not specify access modes are properly ignored now too - this will be relevant for setActorAccess.)
Additionally, I've extracted all references to actorRelations into a single array listing all of them (just two at the time), to make sure that, when we need to take into account additional potential actorRelations, we update all relevant code. This will be especially relevant with setActorAccess, which will make copies of Policies and Rules, and therefore needs to make sure it copies over all relevant relations.

[x] I've added a unit test to test for potential regressions of this bug.
[x] The changelog has been updated, if applicable.
[x] Commits in this PR are minimal and have descriptive commit messages.

@NSeydoux Are you OK with merging this while we await a definitive decision on the desired behaviour? Since #800 builds on this.

This was merged as part of #800.


Feature/graph creation

Brings the following major features into pagi.js:

Graph parsing (xml)
Graph traversal
Graph serialization (xml)

Tests have been added for most of the major functionality. I can flesh out lower level unit tests when I have more time.
Pending features per discussions with Mahesh:

[x] Add .getText on spanContainers.
[x] Remove the previous edge from sequence nodes and rely on a single incoming next edge. This assumes a node can only reside in one sequence.
[x] Add parent edges to nodes within a spanContainer. Provide the .getFirstParentOfType and .getParentsOfType to sequence nodes.

Assuming tests pass, this PR should be good to go. If you see any convenience functions that should be added let me know.

Moving to digitalreasoning/pagijs.


[FR] Add an optional duration

Hi,
I really appriciate the work you did with this plugins. I would like to use it as a periodic reminder to stay focused (currently use an app on my phone for this).
Can you add a duration: x parameter so that the metronome automatically stops after x minutes?
Thanks for your work!

+1 fantastic idea :)

I like this idea too. Often when I use the metronome I only need it when I’m starting off the song so some kind of parameter for automatically stopping it after a period of time would be indeed helpful.

Added in 4d1fa91b2b37d4ebc7549f16b9c95b07f15b52aa and released in 0.8.0. Please update obsidian-metronome-plugin in Obsidian to use this new feature. Thanks!

works great!


Color support for Windows

I assume you're aware due to the try/except on the import, Windows versions of Python do not ship with the curses module. While there are ways to get a Windows-compatible curses module, it's not a reasonable proposition when there are tried-and-trued alternatives to colored output for Python.
Most apps and packages use colorama for colored output, including some colored logging packages that exist.
It's nice that logzero still works fine on Windows. But, colored output is certainly an attractive feature, so I hope you'll consider changing the method by which output is colorized to be cross-platform out-of-the-box.
Thanks for your time and effort.

Hello,
Thanks thats a good feedback, and I'm absolutely willing to accept improvements in this area. I don't have a Windows machine of VM at hand at the moment, so this probably won't be improved by myself soon. But as said, I'm willing to accept and mentor contributions in this regard.
Any thoughts on what's the best colored logging package in your opinion?

Honestly, I had never looked any colored logging packages up before. I just figured it'd be best to link some examples and so searched PyPI for colored logging and linked the top recently created/updated packages.
Another tool I use daily, httpie, uses colorama along with Pygments to tokenize and color its output akin to syntax highlighting. better-exceptions conditionally installs and uses colorama for its coloring.
Unfortunately, I can't justify spending time on this at present. I have too many projects that have long suffered neglect due to a lack of hobby time. I've only just found time for some of that work when I saw this package. I was hoping to drop it into a project but will have to move on without it until I catch up on my backlog of work : (

First of all, thanks for the research and the references!
I will take a look at color support for Windows / colorama integration soon, if nobody else contributes something for this.
I hope you'll try out logzero by dropping it into projects anyway, I think that you'll be happy with it and I'm hoping that you continue to submit feedback and ideas!
Thanks ;)

So, I've installed Windows in a VM and played around with colorama, and it seems to work quite nicely: https://github.com/metachris/logzero/commit/4207194dc3f752b6c6536bba94f31a04a0de23be#diff-27e20f4aad63c2b9a1ce2974b00a4da4
What do you think?

So, I've installed Windows in a VM and played around with colorama, and it seems to work quite nicely. I'm considering to release this in pretty much this form: #20
Any thoughts?

Looks fine to me. Thanks. Seems like I'll get to put this into gmusicapi-scripts for the 1.0.0 release after all : P

I've just merged this into master and released v1.3.0
@thebigmunch can you please confirm that it works for you on Windows?

I had already tested your PR, bit I can give it a post-release test when I get home in a few hours.

Still works well : P

Thanks! Awesome.


net: Avoid calling L2 functions on offloaded interface

Calling functions like net_if_start_dad, join_mcast_nodes or
net_if_start_rs lead to L2 API function calls, which is not correct
for offloaded interfaces and leads to a crash. This is especially
problematic, as they are called in the default configuration.
Avoid calling these functions while an offloaded interface is brought up
by adding an extra jump label.
Signed-off-by: Robert Lubos<EMAIL_ADDRESS>

CI complained about unused label in builds with CONFIG_NET_OFFLOAD disabled. Switched to use IS_ENABLED(CONFIG_NET_OFFLOAD) macro instead of #ifdef to fix this.


⚠ Syncthing-KA-TM has degraded performance

In f3fd643, Syncthing-KA-TM ($URL_SYNCTHING_MERZ_NIMBUS_TM) experienced degraded performance:

HTTP code: 200
Response time: 5762 ms

Resolved: Syncthing-KA-TM performance has improved in 8013e72 after 5 minutes.


lib.d.ts - XMLHttpRequest - send

Current definition allows only string and Document argument. But it should allows BodyInit defined as:
typedef (Blob or BufferSource or FormData or URLSearchParams or USVString) BodyInit;
https://fetch.spec.whatwg.org/#bodyinit
https://xhr.spec.whatwg.org/#the-send()-method

+1

This regressed since 1.4, we need an additional overload:
    send(data?: Document): void;
    send(data?: string): void;
    send(data?: any): void;

@janousek and @d3trax can you give this file a try: https://github.com/Microsoft/TypeScript/blob/master/bin/lib.d.ts

Could you clarify why it's closed? I don't see overload with other BodyInit types and I receive error on latest master trying to pass FormData object to send


Update address size from 32 to 64 bit

Update all types that refer to an address from u32 to u64
Add validation to current targets for values out of range
Clean up some shared Cortex-M code

Signed-off-by: Ryan Fairfax<EMAIL_ADDRESS>

I have validated probe-rs-debugger functionality against ARM devices (STM32H7, RP2040, and nRF52833). Everything works at least as well as before and there is no obvious degradation in performance. This includes flashing, core register values, variable values, breakpoints, stepping, disassembly, RTT, and peripheral registers.

Thanks for the validation - much appreciated


Feedback and ideas for future

assuming https://github.com/LedgerHQ/ledger-node-js-api/pull/26 gets in, here is next step feedback & ideas
JS code

The library have a lot of async. I think it would be interesting to migrate
the promise codebase to async/await (babel compiles it down to Promise
code) This is especially interesting for code that need to chain promises in
sequence (eachSeries) (you can basically do a for loop with a await in
it!).
Apparently new Buffer is deprecated so need to port the remaining usage of
that (and use Buffer.from / Buffer.alloc)
codebase currently have some repetition. Need to think what we could factorize
and if there is a higher level library that could help us on dealing with data
format? (e.g; maybe we should directly manipulate the buffer, but define some
"schema" and inject all data to "serialize" something) – just a thought.

Documentation

https://www.npmjs.com/package/documentation could be interesting to use. We
could generate a API.md that is always in sync with the source code. What we
need to do is to add some "Javadoc" in all public objects/classes/methods. To
have used it before, I'm pretty sure this is compatible with Flow too. (need
to check this works ok with async/await tho)
README should start by telling more what the library is, and how to install
and use it. The part about developing on the project is documented in
CONTRIBUTING.md (and we can have a link to there).
README is a good place to have a simple example so people instantly get the
"gist" of the library. I'm not sure if the whole documentation should stick in
there and maybe we would delegate to this API.md? (we have have links to that
API doc)
Contributing needs doc on how to deploy a new version on npm.

API
These are breaking API changes suggestions (only to be in a major release)

I think it would be better to directly export the classes (LedgerBtc /
LedgerEth / ..) under these names.
Maybe we should have LedgerComm exported as either LedgerCommU2F or
LedgerCommNode. So we can just write the exact same code on Node or on
Browser (and it also simplify documenting this).
I don't think we should suffix any method with _async. It is common to have
promise returned in JS libs. Just need users to know everything is async
(RTFM!), but also if they use Flowtype, you won't have surprises ;)

More general question & thoughts

This is unclear to me what is the lifecycle of a comm and with a
LedgerBtc/LedgerEth instance. E.g. nor the API, nor any runtime check,
prevents me from using comm on 2 instances of these (which I guess is an
issue, especially that instanciating a LedgerBtc will set a state on comm
(setScrambleKey)). Which led me to the question: can we afford creating one
comm at a time & per LedgerBtc/Eth usage? If so, shouldn't this be hidden for
the user? (aka you would do: LedgerBtc.create().then(btc => ...) without
having to worry about comm ?). If we still want comm to have a bigger
lifecycle (like you instanciate it only once for the all app lifecycle), I
think we will need a "context" concept that we bind again that context
when we enter a new call (e.g. like OpenGL programs) and maybe we even need
some ways to throw if user is calling something and we're in the middle of
something else (to avoid concurrent access).

Tests
I suggest we could use Jest engine for tests.

We could use test coverage to determine which things remain to be tested.
(jest provides this)
to test LedgerCommU2F, we could mock implementation of U2F and back it by
node-hid. (jest have nice ways to .mock things). (we still would have browser
example/test for a end-to-end check, but if the mock implementation is
correct, we shouldn't worry much as it would just mean testing the actual
browser U2F)
tests can use async/await too (I think Jest works with async test too)
This would still requires the user to plug a device and manually check&accept
the things on the display. This is nice to have but I think we should also
have a headless version (device simulator) that eventually runs on Travis. I
imagine there will be some addition testing code for the simulator too, like
you want to test both side (the client side and the "device" side) e.g; that
it display something / a given string / ...

CI

as said above, if we can have a headless version (device simulator) we can
eventually run tests on Travis!
but also, we should already setup Travis check that will call the lint and
flow commands (and obviously we connect it to incoming PRs).

It would be great to be able to query an xpub/list of xpubs. Right now the solution I'm applying to get an xpub for some path (let's say  p = m/49'/0'/0'/0/0):

Get the public key & chain code at p.
Get the public key at the parent node of p (m/49'/0'/0'/0/).
Compress p's public key, take HASH160(parent_key)[:4] as the fingerprint, & serialize the rest of the xpub from p's information.

I think it would also be intuitive to make a distinction between coin-specific functionality & general cryptographic functionality. For instance, the BIP32 public key derivation code is applicable for all secp256k1 coins. I understand that the ledger itself has separate apps for each coin type so this might not be possible to separate, but that would make more sense semantically if it's possible.


Input element do not get style applied when use scoped style.

Framework7-Vue version:
0.9.2
Sample code:
<template lang="pug">
f7-list
  f7-list-item
    f7-input(type="text")
</template>

<style scoped>
input {
  color: red;
}
</style>

Should be:
Text in input is RED
Actually is:
Text in input is still BLACK like default.

There is a bug in corresponding render functions.

Such weak selector can't override default styles. Use more strict selector


Question:

Hi
Iam trying to implement playwright-bdd in to an existing framework which was using playwright and cucumber as the test runner using commons as the module.Iam seeing compilation errors when tried to run bdd gen command.Do we have any examples or sample frameworks running on commonjs?

Hi! Yes, here is an example of using playwright-bdd with common js: https://github.com/vitalets/playwright-bdd-example

Hi! Yes, here is an example of using playwright-bdd with common js: https://github.com/vitalets/playwright-bdd-example

Thanks for quick reply. Also have another question do playwright-bdd has the support for other cucumber features like generation of the rerun files and the json report generator.

Also have another question do playwright-bdd has the support for other cucumber features like generation of the rerun files and the json report generator.

Yes for both:

you can rerun last failed tests with Playwright's --last-failed option:npx bddgen && npx playwright test --last-failed

Cucumber JSON reporter is supported.

Also have another question do playwright-bdd has the support for other cucumber features like generation of the rerun files and the json report generator.

Yes for both:

you can rerun last failed tests with Playwright's --last-failed option:

npx bddgen && npx playwright test --last-failed

Cucumber JSON reporter is supported.

Thanks for the quick reply.
One more question i have is. Do we have an equivalent of "scenario.result.status, scenario.pickle.name"
This is really helping me
Thanks in Advance

One more question i have is. Do we have an equivalent of "scenario.result.status, scenario.pickle.name"

Cucumber scenario = Playwright test.
So, scenario.result.status is actually test status and scenario.pickle.name = scenario.name = test.title.


N+1 for Poisson expectation

We have recently had a pre-fit Asimov pull due to the fact that the expected Poisson rate is set to N+1 in
https://github.com/cms-analysis/HiggsAnalysis-CombinedLimit/blob/development/python/ModelTools.py#L169
It's unclear (to me) why exactly this is the case, but it's like that since 2011.

From @gpetruc on 12 Dec 2014:

So, that N+1 vs N is that in practice for gmN combine makes a pre-fit Asimov with the mean expected background instead of the mode (which is what's normally done for Asimov)
Now, if N is small, which is the case for which the difference matters, one is probably outside the asymptotic regime, so the Asimov isn't necessarily the good answer, especially given that outside the asymptotic regime median, mean and mode of the expected distribution of any quantity like limit or significance can differ substantially.
That said, i agree that it is somewhat inconsistent, so maybe we should change it.

So, for a Poisson with mean lambda one has:

the mode is <= lambda.
the median is ~ lambda+1/3 - 0.02/lambda, which clearly breaks for lambda=0 (as does the notion of Poisson mean).

Is the args[0]+1 in https://github.com/cms-analysis/HiggsAnalysis-CombinedLimit/blob/development/python/ModelTools.py#L169 an attempt at the median?
Anyway, from what I have understood of Poisson medians, I would propose - if that is the relevant quantity for Asimovs:

mean=0 -> median = 0
mean>0 -> median = Floor[mean + 1/3 - 0.02/mean]

Hi André,
When considering the nuisance parameter, you should be looking at a Gamma
distribution, not a Poisson.
The Poisson is for the event yield.
Giovanni
Il 24-lug-2015 13:10, "André David"<EMAIL_ADDRESS>ha scritto:

From @gpetruc https://github.com/gpetruc on 12 Dec 2014:
So, that N+1 vs N is that in practice for gmN combine makes a pre-fit
Asimov with the mean expected background instead of the mode (which
is what's normally done for Asimov)
Now, if N is small, which is the case for which the difference matters,
one is probably outside the asymptotic regime, so the Asimov isn't
necessarily the good answer, especially given that outside the asymptotic
regime median, mean and mode of the expected distribution of any quantity
like limit or significance can differ substantially.
That said, i agree that it is somewhat inconsistent, so maybe we should
change it.
So, for a Poisson with mean lambda one has:

the mode is <= lambda.
the median is ~ lambda+1/3 - 0.02/lambda, which clearly breaks for
lambda=0 (as does the notion of Poisson mean).

Is the args[0]+1 in
https://github.com/cms-analysis/HiggsAnalysis-CombinedLimit/blob/development/python/ModelTools.py#L169
an attempt at the median?
Anyway, from what I have understood of Poisson medians, I would propose -
if that is the relevant quantity for Asimovs:

mean=0 -> median = 0
mean>0 -> median = Floor[mean + 1/3 - 0.02/mean]

—
Reply to this email directly or view it on GitHub
https://github.com/cms-analysis/HiggsAnalysis-CombinedLimit/issues/161#issuecomment-124477155
.

Closing the issue since this is now a known feature

I want to revisit this topic, as it seems there is still an inconsistency motivated by the fact that a gmN nuisance should really be consistent with a rateParam plus control region setup. Take for example the cards:
card_CR.txt:
imax 2 number of bins
jmax 2 number of processes minus 1
kmax 1 number of nuisance parameters
---------------------------------------
shapes * * FAKE
---------------------------------------
bin          SR    CR
observation  12    10
---------------------------------------
bin          SR    SR    SR     CR
process     sig  bkg1  bkg2   bkg2
process       0     1     2      1
rate          5    10     5     10
---------------------------------------
lumi lnN   1.02  1.02  1.02      -
---------------------------------------
bkg2_norm rateParam * bkg2 1.0 [0,2]

and card_gmN.txt:
imax 1 number of bins
jmax 2 number of processes minus 1
kmax 2 number of nuisance parameters
---------------------------------------
shapes * * FAKE
---------------------------------------
bin          SR
observation  12
---------------------------------------
bin                 SR    SR    SR
process            sig  bkg1  bkg2
process              0     1     2
rate                 5    10     5
---------------------------------------
lumi lnN          1.02  1.02  1.02
bkg2_norm gmN 10     -     -   0.5

These two cards construct identical models (apart from the scaling of bkg2_norm by 10 in gmN case). This was confirmed by evaluating the DeltaNLL for each model with a random grid of points in the 3d parameter space. The max difference was less than 1e-14.
 A scan:
combine -M GenerateOnly -t -1 card_gmN.root -n .gmNasimov --saveToys
combine -M MultiDimFit card_gmN.root -n .gmNscan \
    -D higgsCombine.gmNasimov.GenerateOnly.mH120.123456.root:toys/toy_asimov --algo grid --rMin 0 --rMax 0.5
combine -M GenerateOnly -t -1 card_CR.root -n .CRasimov --saveToys
combine -M MultiDimFit card_CR.root -n .CRscan \
    -D higgsCombine.CRasimov.GenerateOnly.mH120.123456.root:toys/toy_asimov --algo grid --rMin 0 --rMax 0.5

shows more clearly

We can recover the CR card behavior by pre-setting the nuisance parameter: combine -M AsymptoticLimits --run blind card_gmN.root --setParameters bkg2_norm=10.
Of course, all observed quantities are immune to such problems, and the post-fit expected is also immune since the Asimov is generated with an appropriate value of bkg2_norm, i.e.
combine -M AsymptoticLimits card_CR.root
combine -M AsymptoticLimits card_gmN.root

produce identical results.
Aside from the inconsistency, there may be a question as to which setup is more accurate w.r.t. toys. I tried to evaluate this by running
combine -M HybridNew card_CR.root --LHCmode LHC-limits -t -1 --expectedFromGrid 0.5
combine -M HybridNew card_gmN.root --LHCmode LHC-limits -t -1 --expectedFromGrid 0.5
combine -M HybridNew card_gmN.root --LHCmode LHC-limits -t -1 --setParameters bkg2_norm=10  --expectedFromGrid 0.5

each with various settings of -toysH, --interpAcc, and --rAbsAcc, but I could not find a way to lower the numeric uncertainty enough to show a clear relation between the three setups. In general, all three are close to each other and around 1.8something

I agree with @nsmith- here that the two setups should be identical, and with that understanding, it does seem that this should be fixed. Is there something I am missing? If we agree on it, its extremely easy to implement.

Hi all,
I think we should be a bit careful here. In the land of low statistics,
even the default toy based methods may not be correct. Bob has a paper all
about this specifically in the on/off problem (where the gamma arises) and
if one uses our "cousins-highland" approach for the nuisance parameter,
then its default value shouldn't matter (the observed value does as we
use that to construct the likelihood).  It might make more sense that we
ask people to use toys in a more careful way when we are in low-statistics
regimes where this make a difference. Basically, I think neither Asimov nor
our standard toy based approaches are really correct when the difference
between N and N+1 is significant.
Nick
On Thu, Nov 2, 2023 at 3:22 PM kcormi @.***> wrote:

I agree with @nsmith- https://github.com/nsmith- here that the two
setups should be identical, and with that understanding, it does seem that
this should be fixed. Is there something I am missing? If we agree on it,
its extremely easy to implement.
—
Reply to this email directly, view it on GitHub
https://github.com/cms-analysis/HiggsAnalysis-CombinedLimit/issues/161#issuecomment-1790947573,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAMEVW5LEQNTJTRENAE2CJDYCO3DXAVCNFSM4AX5KLLKU5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TCNZZGA4TINZVG4ZQ
.
You are receiving this because you modified the open/close state.Message
ID: @.***
.com>

I would characterize this issue is more about the consistency of two ways of developing a model than whether or not the usual statistical methods work well for such a model.

Hi all,
I was wondering about something I am seeing with the gmN parameter that may also be related to this N+1 issue.
Consider the following cards (Card1 and Card2 respectively) where the only difference is the addition of the gmN parameter in Card2:
imax 1 number of bins
jmax 2 number of processes minus 1
kmax 1 number of nuisance parameters
---------------------------------------
shapes * * FAKE
---------------------------------------
bin          SR
observation  12
---------------------------------------
bin                 SR    SR    SR
process            sig  bkg1  bkg2
process              0     1     2
rate                 5    10     5
---------------------------------------
lumi lnN          1.02  1.02  1.02

imax 1 number of bins
jmax 2 number of processes minus 1
kmax 2 number of nuisance parameters
---------------------------------------
shapes * * FAKE
---------------------------------------
bin          SR
observation  12
---------------------------------------
bin                 SR    SR    SR
process            sig  bkg1  bkg2
process              0     1     2
rate                 5    10     5
---------------------------------------
lumi lnN          1.02  1.02  1.02
bkg2_norm gmN 10     -     -   0.5

executing combine -M Significance card*.txt --expectSignal=1 -t -1 on these returns two different expected significances, but not in the direction I would expect.
For Card1: 1.22374
For Card2: 1.22432
The expected significance actually increases with the addition of the gmN systematic which I would not expect.

I think we do something for the scaling in case of a gamma parameter.
See the likelihood scan from combine, vs implementing it in python as two separate bins.
The one of combine seems to cross, which indicates that the nuisance is more affecting the signal, rather than the background. But honestly, I don't remember how exactly is implemented.
Does somebody have already the same dc implemented w/ rateparams?

(command)
combine -M MultiDimFit --algo=grid --rMin=-2 --rMax=4 -t -1 --expectSignal=1 dc2.txt -n dc2

Does somebody have already the same dc implemented w/ rateparams?

scroll up :)

:+1: (actually, I should have read the thread in more details, as apparently is on something else :D ).
I think there is indeed a mistake in the prefit assignment that only reflect on the gmN Asimov. (as said above)
It is true that we need to think in terms of gamma distribution, as Giovanni said, but it is worth to notice that the gamma is a probability distribution for the parameter and not the likelihood itself.
(notation, the probability that I mess up with it is 1 😉 )
In particular, being $p( \vec{x} | \vec{p})$ a probability distribution on $\vec{x}$ with parameters $\vec{p}$ (now the vector drops),  and being $\mathcal{P}( d | \lambda)$ the Poisson distribution and $\Gamma(x | k,\theta)$ the gamma distribution.
what we want as likelihood is (i.e., a CR from which is derived the measurement):
$\mathcal{L} = \mathcal{P}(D| r s + \dots + \mu \tau) \mathcal{P}(d| \mu)$ (1)
where $D$, $d$ are the observed data in the SR, and CR, $\mu$ is the background normalization and $\tau$ is a transfer factor. (you can put it either on the left or on the right)
now it is known [ Ref. 1 ] that the Bayesian posterior of the of (1) with an uniform prior on the parameter is the Gamma distribution.
(Just look at the functional form of the gamma and the Poisson and you see that you need to exchange a parameter with the variable and rearrange the parameter values).
In this case we can write our likelihood function as (we need just to match what is what between the different terms):
$\mathcal{L} = \mathcal{P}(D | r s \dots + \mu\tau) \Gamma(\mu\tau |d+1, \tau )$
Note again that in the $\Gamma$ have the parameter in the probability space, as we applied Bayes theorem.
For what concern us, I think we should make sure that the prefit Asimov is indeed identical and not shifted. In the end, we are choosing the first implementation with the Poisson (though they are equivalent).
You can check that the formulas above yield the same result with some numerical calculation.
For example: (adding a small vertical shift between the two for plotting, $-\log\mathcal{L}$, $\mu$)


Missing documentation for Argument flattenFolder

Hi,
I'm not finding any documentation for Argument flattenFolder, you may want to add it.
Thanks,
Sam

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 81f02378-b85b-d3a4-d3c8-f07d935367da
Version Independent ID: fab57f76-35d2-4024-6f7f-1740d7b24743
Content: Copy Files task - Azure Pipelines & TFS
Content Source: docs/pipelines/tasks/utility/copy-files.md
Product: devops
GitHub Login: @andyjlewis
Microsoft Alias: alewis

Ah, yes. Thanks @propellor . This is the description in the product: "Flatten the folder structure and copy all files into the specified target folder." I'll add it to the docs :)

However, flatten folder is not available in TFS 2015, so TFS 2015 version of this page should not show it


title: added blog on flask api authentication

Checklist
Please note that in case of below checklist not updated accordingly, the maintainers have the right to immediately close your pull request.

[x] You created a branch from dev branch.
[x] You PR is raised on dev branch and not onmaster.
[x] Blog follows Blog Guidelines.
[x] You have read the Contribution Guidlines before creating this PR.
[x] Checked for any duplicate blog entry.
[x] Checked for all grammatical errors in the blog.
[x] You have not published the blog content anywhere else on the internet on platforms like medium, dev.to etc.
[x] The content is not copy-pasted from anywhere else on the Internet.

Description of change
Author Details for Blog Post Entry
id: Solomon Esenyi
bio: Python Developer and Technical Writer with a passion for open-source, cryptography, and serverless technologies.
github: LordGhostX
linkedin: solomon-esenyi
twitter: LordGhostX

@pareek-narendra @puneetsingh24 is there anything I need to do for the article again?

@pareek-narendra

@LordGhostX We'll review this in some time and get back to you if changes are necessary

Hi @LordGhostX
First things first, apologies for the delayed review.
We have reviewed it and is missing the following from the initially discussed outline:

Registered user login
Non-registered user login
Implementing bearer token expiration time

Please try adding/creating content for the above sections. It will provide more comprehensive value for the readers.
Additionally, please add this demo code in this repo by raising a pull request: https://github.com/LoginRadius/engineering-blog-samples
Thank you,
@raghunath-r-a

Closing this due to no response.


PSInline script- Failed

I was trying to run below PS Inline script thru Azure devops Pipeline. But iam getting error on agent log
Pipeline code:
trigger:

master

pool:
name: 'Dev1'
steps:

task: PowerShell@2
inputs:
targetType: 'inline'
script: |
# Write your PowerShell commands here.
New-Item -Path "C:\Manoj" -Force

Error on Agent:
Starting: PowerShell
Task         : PowerShell
Description  : Run a PowerShell script on Linux, macOS, or Windows
Version      : 2.165.0
Author       : Microsoft Corporation
Help         : https://docs.microsoft.com/azure/devops/pipelines/tasks/utility/powershell
Generating script.
========================== Starting Command Output ===========================
"C:\windows\System32\WindowsPowerShell\v1.0\powershell.exe" -NoLogo -NoProfile -NonInteractive -ExecutionPolicy Unrestricted -Command ". 'C:\tmp\vsts-agent-win-x64-2.165.2_work_temp\65dd1488-c132-4b9a-8403-0604d37f43a4.ps1'"
New-Item : Access to the path 'C:\Manoj' is denied.
At C:\tmp\vsts-agent-win-x64-2.165.2_work_temp\65dd1488-c132-4b9a-8403-0604d37f43a4.ps1:4 char:1

New-Item -Path "C:\Manoj" -Force

  + CategoryInfo          : PermissionDenied: (C:\Manoj:String) [New-Item], UnauthorizedAccessException
  + FullyQualifiedErrorId : NewItemUnauthorizedAccessError,Microsoft.PowerShell.Commands.NewItemCommand

##[error]PowerShell exited with code '1'.
Finishing: PowerShell

Thanks for your question. It looks like you're working through an issue with your scenario or implementation, rather than an issue with the documentation. Here are a couple of options where you might consider asking your question:

Azure DevOps Support Bot
Azure DevOps on Stack Overflow


Error: colorama import just_fix_windows_console   under Ubuntu 20.04

Hello,
would you please update the code so it can be used under Linux, in my case Ubuntu 20.04. The problem is, Windows specific code causing the following error because it's not recognizing Linux:
%%% python3 PlanformCreator2.py
Traceback (most recent call last):
File "/home/klaus/Programme/PlanformCreator2SC/PlanformCreator2.py", line 47, in
from common_utils       import *
File "/home/klaus/Programme/PlanformCreator2SC/modules/common_utils.py", line 9, in
from colorama import just_fix_windows_console
ImportError: cannot import name 'just_fix_windows_console' from 'colorama' (/usr/lib/python3/dist-packages/colorama/init.py)
The fix is probably wrapping the function according to: https://docs.python.org/3/library/sys.html#sys.platform  so it's omitted when running the software under Linux.
Thank you!

Hello Klaus,
thank you for this issue!
... this is strange since up to now this problem didn't arise...
Anyway - I made a fix and just pushed it. Could you "clone" again and check, if it works?
Thank you!

Hello,
it works!
Question: Is there a way/trick to define a constant trailing edge thickness e.g. 0.5mm in the sense of "design for manufacturing"

Hello Klaus,
... supi! (thank you for testing)
--
Yes, you can set the te gap in mm when exporting based on the local chord:

(te gap can also be set in 'dxf export')

Bug fixed


feed needs empty state for newly registered users

This just needs a design. Do you have a screenshot for how it looks right now?

no screenshot but I think it was simply path Are.na/Feed and nothing else. Same design as profile and channel empty states?


refactor: Renaming around Protogalaxy Prover

This is literally just a ton of renaming. Some highlights:

I wanted to stop using the term "optimised" around the Pg optimization that skips computing zero because we have many optimizations and the name was unclear. I also put the extra qualifier on the non-production case where we don't use that optimization, which is currently just in tests of the combiner.
Ariel told me that the real name of the protocol is Protogalaxy--this is in the name of the paper in pdf form, but not on the eprint page :shrug:
PowPolynomial is now a misnomer because it doesn't involve powers of a challenge $\beta$, but rather a bunch of monomials generate from a set of $\beta_i$'s. I didn't have to change this but I do think clarity here comes at a low enough cost to merit making the change.
We have functions that refer to the "full honk" relation, which is fine, but strictly speaking there is no connection to Honk when using Protogalaxy or sumcheck in insolation, so it makes sense to give more agnostic and IMO slightly clearer names.

Benchmark results
Metrics with a significant change:

proof_construction_time_sha256_30_ms (16): 1,422 (-18%)
protocol_circuit_simulation_time_in_ms (private-kernel-tail-to-public): 664 (-51%)
avm_simulation_time_ms (Token:_increase_public_balance): 40.1 (-46%)

Detailed results
All benchmarks are run on txs on the Benchmarking contract on the repository. Each tx consists of a batch call  to create_note and increment_balance, which guarantees that each tx has a private call, a nested private call, a public call, and a nested public call, as well as an emitted private note, an unencrypted log, and public storage read and write.
This benchmark source data is available in JSON format on S3 here.
Proof generation
Each column represents the number of threads used in proof generation.

Metric
1 threads
4 threads
16 threads
32 threads
64 threads

proof_construction_time_sha256_ms
5,755
1,556
714
744
771 (-1%)

proof_construction_time_sha256_30_ms
11,509 (-1%)
3,100
:warning: 1,422 (-18%)
1,504 (-5%)
1,563 (+2%)

proof_construction_time_sha256_100_ms
44,162
12,224 (+4%)
5,835 (+8%)
5,906 (+9%)
6,141 (+3%)

proof_construction_time_poseidon_hash_ms
78.0
34.0
34.0
57.0 (+2%)
88.0 (+1%)

proof_construction_time_poseidon_hash_30_ms
1,528
421
202
233 (+5%)
274 (+1%)

proof_construction_time_poseidon_hash_100_ms
5,635
1,514
671
732
745 (-1%)

L2 block published to L1
Each column represents the number of txs on an L2 block published to L1.

Metric
4 txs
8 txs
16 txs

l1_rollup_calldata_size_in_bytes
4,356
7,876
14,884

l1_rollup_calldata_gas
50,256
92,924
178,240

l1_rollup_execution_gas
842,916
1,576,480
3,361,852

l2_block_processing_time_in_ms
258 (+4%)
456 (+1%)
820 (-4%)

l2_block_building_time_in_ms
11,370
22,151 (-1%)
44,437

l2_block_rollup_simulation_time_in_ms
11,370
22,150 (-1%)
44,437

l2_block_public_tx_process_time_in_ms
9,657
20,336 (-1%)
42,584

L2 chain processing
Each column represents the number of blocks on the L2 chain where each block has 8 txs.

Metric
3 blocks
5 blocks

node_history_sync_time_in_ms
3,054
3,978 (+1%)

node_database_size_in_bytes
12,615,760
16,687,184

pxe_database_size_in_bytes
16,254
26,813

Circuits stats
Stats on running time and I/O sizes collected for every kernel circuit run across all benchmarks.

Circuit
simulation_time_in_ms
witness_generation_time_in_ms
input_size_in_bytes
output_size_in_bytes
proving_time_in_ms

private-kernel-init
98.4 (+1%)
396 (+4%)
21,735
44,860
N/A

private-kernel-inner
177 (+1%)
701 (+3%)
72,544
45,007
N/A

private-kernel-reset-tiny
314 (+2%)
709 (+3%)
65,593
44,846
N/A

private-kernel-tail
166 (-1%)
135 (+1%)
50,644
52,257
N/A

base-parity
5.64 (+1%)
N/A
160
96.0
N/A

root-parity
35.5 (+1%)
N/A
73,948
96.0
N/A

base-rollup
3,431 (-1%)
N/A
189,136
664
N/A

block-root-rollup
41.5 (+1%)
N/A
58,205
2,448
N/A

public-kernel-setup
84.5
N/A
105,085
71,222
N/A

public-kernel-app-logic
96.6 (-1%)
N/A
104,911
71,222
N/A

public-kernel-tail
859
N/A
390,582
16,414
N/A

private-kernel-reset-small
311 (+2%)
N/A
66,341
45,629
N/A

private-kernel-tail-to-public
:warning: 664 (-51%)
610 (+3%)
455,400
1,825
N/A

public-kernel-teardown
84.4
N/A
105,349
71,222
N/A

merge-rollup
20.3
N/A
38,174
664
N/A

undefined
N/A
N/A
N/A
N/A
78,174 (-12%)

Stats on running time collected for app circuits

Function
input_size_in_bytes
output_size_in_bytes
witness_generation_time_in_ms

ContractClassRegisterer:register
1,344
11,731
343

ContractInstanceDeployer:deploy
1,408
11,731
18.3 (+1%)

MultiCallEntrypoint:entrypoint
1,920
11,731
407 (+1%)

FeeJuice:deploy
1,376
11,731
389

SchnorrAccount:constructor
1,312
11,731
75.2 (+4%)

SchnorrAccount:entrypoint
2,304
11,731
394

Token:privately_mint_private_note
1,280
11,731
104 (+3%)

FPC:fee_entrypoint_public
1,344
11,731
25.9 (+6%)

Token:transfer
1,312
11,731
228 (+4%)

Benchmarking:create_note
1,344
11,731
85.8

SchnorrAccount:verify_private_authwit
1,280
11,731
32.1 (+17%)

Token:unshield
1,376
11,731
534 (+2%)

FPC:fee_entrypoint_private
1,376
11,731
751 (+6%)

AVM Simulation
Time to simulate various public functions in the AVM.

Function
time_ms
bytecode_size_in_bytes

FeeJuice:_increase_public_balance
60.3 (+4%)
7,101

FeeJuice:set_portal
10.6 (-12%)
2,128

Token:constructor
83.2
25,285

FPC:constructor
53.1 (-4%)
17,853

FeeJuice:mint_public
39.5 (+1%)
5,415

Token:mint_public
45.5 (-1%)
10,101

Token:assert_minter_and_mint
66.2 (-1%)
6,844

AuthRegistry:set_authorized
41.4 (+9%)
3,969

FPC:prepare_fee
230 (+1%)
6,747

Token:transfer_public
21.6 (+6%)
38,126

FPC:pay_refund
49.5 (-1%)
9,398

Benchmarking:increment_balance
1,193 (-1%)
6,179

Token:_increase_public_balance
:warning: 40.1 (-46%)
7,705

FPC:pay_refund_with_shielded_rebate
64.1 (-1%)
9,881

Public DB Access
Time to access various public DBs.

Function
time_ms

get-nullifier-index
0.159 (-5%)

Tree insertion stats
The duration to insert a fixed batch of leaves into each tree type.

Metric
1 leaves
16 leaves
64 leaves
128 leaves
256 leaves
512 leaves
1024 leaves

batch_insert_into_append_only_tree_16_depth_ms
2.20 (-2%)
3.99 (-1%)
N/A
N/A
N/A
N/A
N/A

batch_insert_into_append_only_tree_16_depth_hash_count
16.8
31.7
N/A
N/A
N/A
N/A
N/A

batch_insert_into_append_only_tree_16_depth_hash_ms
0.115 (-2%)
0.113 (-1%)
N/A
N/A
N/A
N/A
N/A

batch_insert_into_append_only_tree_32_depth_ms
N/A
N/A
11.5 (+1%)
17.7 (-3%)
30.9 (-5%)
59.5 (-1%)
113 (-4%)

batch_insert_into_append_only_tree_32_depth_hash_count
N/A
N/A
95.9
159
287
543
1,055

batch_insert_into_append_only_tree_32_depth_hash_ms
N/A
N/A
0.110 (+2%)
0.103 (-3%)
0.101 (-4%)
0.102 (-1%)
0.102 (-3%)

batch_insert_into_indexed_tree_20_depth_ms
N/A
N/A
14.4 (+1%)
25.9 (-2%)
43.6 (-4%)
83.1
161 (-5%)

batch_insert_into_indexed_tree_20_depth_hash_count
N/A
N/A
109
207
355
691
1,363

batch_insert_into_indexed_tree_20_depth_hash_ms
N/A
N/A
0.110 (+1%)
0.105 (-1%)
0.106 (-4%)
0.103
0.103 (-4%)

batch_insert_into_indexed_tree_40_depth_ms
N/A
N/A
16.6
N/A
N/A
N/A
N/A

batch_insert_into_indexed_tree_40_depth_hash_count
N/A
N/A
132
N/A
N/A
N/A
N/A

batch_insert_into_indexed_tree_40_depth_hash_ms
N/A
N/A
0.106
N/A
N/A
N/A
N/A

Miscellaneous
Transaction sizes based on how many contract classes are registered in the tx.

Metric
0 registered classes
1 registered classes

tx_size_in_bytes
64,779
668,997

Transaction size based on fee payment method
| Metric |  |
| - |  |

Looks like git didn't detect me moving pow.hpp


Use snapcraft-preload to redirect shm access

One can rather use the private shared memory interface to allow this.
plugs:
  shared-memory:
    private: true

Superceded by #21


KOGITO-8237: [SWF Editor] Add support to Async API

EPIC: https://issues.redhat.com/browse/KOGITO-8237
JIRA issues:
https://issues.redhat.com/browse/KOGITO-8376
https://issues.redhat.com/browse/KOGITO-8533

I am still testing this PR but I found some places where we should probably generalize or mention AsyncApi. There are mentions only for OpenApi:

https://github.com/kiegroup/kie-tools/blob/main/packages/serverless-logic-web-tools/src/settings/serviceRegistry/ServiceRegistrySettingsTab.tsx#L96
https://github.com/kiegroup/kie-tools/blob/main/packages/serverless-logic-web-tools/src/settings/serviceRegistry/ServiceRegistrySettingsTab.tsx#L140
https://github.com/kiegroup/kie-tools/blob/main/packages/serverless-logic-web-tools/src/settings/serviceAccount/ServiceAccountSettingsTab.tsx#L141
https://github.com/kiegroup/kie-tools/blob/main/packages/vscode-extension-serverless-workflow-editor/README.md?plain=1#L30
https://github.com/kiegroup/kie-tools/blob/main/packages/vscode-extension-serverless-workflow-editor/README.md?plain=1#L32
https://github.com/kiegroup/kie-tools/blob/main/packages/vscode-extension-serverless-workflow-editor/package.json#L198
https://github.com/kiegroup/kie-tools/blob/main/packages/vscode-extension-serverless-workflow-editor/package.json#L203
https://github.com/kiegroup/kie-tools/blob/main/packages/serverless-logic-web-tools/src/editor/Deploy/ConfirmOptions/SwfDeployOptions.tsx#L112

I am still testing this PR but I found some places where we should probably generalize or mention AsyncApi. There are mentions only for OpenApi:

https://github.com/kiegroup/kie-tools/blob/main/packages/serverless-logic-web-tools/src/settings/serviceRegistry/ServiceRegistrySettingsTab.tsx#L96
https://github.com/kiegroup/kie-tools/blob/main/packages/serverless-logic-web-tools/src/settings/serviceRegistry/ServiceRegistrySettingsTab.tsx#L140
https://github.com/kiegroup/kie-tools/blob/main/packages/serverless-logic-web-tools/src/settings/serviceAccount/ServiceAccountSettingsTab.tsx#L141
https://github.com/kiegroup/kie-tools/blob/main/packages/vscode-extension-serverless-workflow-editor/README.md?plain=1#L30
https://github.com/kiegroup/kie-tools/blob/main/packages/vscode-extension-serverless-workflow-editor/README.md?plain=1#L32
https://github.com/kiegroup/kie-tools/blob/main/packages/vscode-extension-serverless-workflow-editor/package.json#L198
https://github.com/kiegroup/kie-tools/blob/main/packages/vscode-extension-serverless-workflow-editor/package.json#L203
https://github.com/kiegroup/kie-tools/blob/main/packages/serverless-logic-web-tools/src/editor/Deploy/ConfirmOptions/SwfDeployOptions.tsx#L112

Thanks @tomasdavidorg for pointing these. I made these changes yesterday. :)


Feature: Use case 2a

This becomes use case 2a. Started with a copy of use case 10.

[x] Add technologies
[x] Run optimisation
[ ] Update results_template
[ ] Bring results into results_template

As decided after talking it uses shorter filenames (without use case name as a suffix).

The update function does not work yet. Values missing.

Optimization with technologies "nuclear-st" and "ch4-gt" runs so far. Also tested manually with a lower marginal cost for "ch4-gt" in one country to force this plants to operate prior to nuclear plants. Works.

Next, I will update to the new structure and naming rules introduced by #46 via merging with dev.

Next, I will adopt the postprocessing step to the updated results_template.

Commit https://github.com/modex-flexmex/oemo-flex/pull/44/commits/e7578bed25f3c428ed31fd7f48fab78be469a8ed should be seen as a workaround until https://github.com/modex-flexmex/oemo-flex/issues/49 is solved via input data.

So far, nuclear plants are expanded, CH4 gas turbines are not, as can be seen in /oemo-flex/experiment_1/003_results_optimization/FlexMex1_2a/capacities.csv.

Looks like this renaming is one and the same and the very same renaming for every use case lies ahead of us now if we want to change only one of these function names again! This speaks in favour of a more object-oriented approach!

Something similar is true for the last commits. We will have to do this again and again.

I get the following error:
20:12:29-INFO-Optimization successful...
20:12:56-INFO-Writing the results to /home/unndreay/Workspaces/oemo-flex/experiment_1/003_results_optimization/FlexMex1_2a
20:12:57-INFO-Restoring attributes will overwrite existing attributes.
Traceback (most recent call last):
  File "/home/unndreay/Workspaces/oemo-flex/experiment_1/scripts/FlexMex1_2a/runall.py", line 18, in <module>
    postprocessing.main()
  File "/home/unndreay/Workspaces/oemo-flex/experiment_1/scripts/FlexMex1_2a/postprocessing.py", line 61, in main
    summed_sequences = get_summed_sequences(sequences_by_tech, prep_elements)
  File "/home/unndreay/Workspaces/oemo-flex/oemoflex/postprocessing.py", line 193, in get_summed_sequences
    df = prep_elements[tech_carrier][basic_columns]
KeyError: 'uranium-nuclear'

Applies to mapping.csv (not included in this repo): Changed lines like
EnergyConversion_Capacity_Electricity_Nuclear_ST,MW,uranium-nuclear,uranium,dispatchable,nuclear,capacity

into
EnergyConversion_Capacity_Electricity_Nuclear_ST,MW,uranium-nuclear-st,uranium,dispatchable,nuclear-st,capacity

(added -st to uranium-nuclear and nuclear)

Changed type from 'dispatchable' to 'conversion' for ch4-gt in mapping.csv, too (not part of the repo).

Tried to manually introduce negative_gradient and positive_gradient to translate "MaxPowerChange" parameter from Scalars.csv. An issue came up: #57.
More over the parameters do not work in combination with greenfield optimization:
19:24:34-INFO-Creating EnergySystem from datapackage
19:26:07-INFO-Creating the optimization model
Traceback (most recent call last):
  File "/home/unndreay/Workspaces/oemo-flex/experiment_1/scripts/FlexMex1_2a/runall.py", line 16, in <module>
    optimization.main()
  File "/home/unndreay/Workspaces/oemo-flex/experiment_1/scripts/FlexMex1_2a/optimization.py", line 42, in main
    m = Model(es)
  File "/home/unndreay/.virtualenvs/oemo-flex/lib/python3.7/site-packages/oemof/solph/models.py", line 267, in __init__
    super().__init__(energysystem, **kwargs)
  File "/home/unndreay/.virtualenvs/oemo-flex/lib/python3.7/site-packages/oemof/solph/models.py", line 98, in __init__
    self._construct()
  File "/home/unndreay/.virtualenvs/oemo-flex/lib/python3.7/site-packages/oemof/solph/models.py", line 105, in _construct
    self._add_child_blocks()
  File "/home/unndreay/.virtualenvs/oemo-flex/lib/python3.7/site-packages/oemof/solph/models.py", line 133, in _add_child_blocks
    block._create(group=self.es.groups.get(group))
  File "/home/unndreay/.virtualenvs/oemo-flex/lib/python3.7/site-packages/oemof/solph/blocks.py", line 142, in _create
    f.negative_gradient['ub'][t] * f.nominal_value)
TypeError: unsupported operand type(s) for *: 'int' and 'NoneType'

With capacities set there is no error anymore.
Additionally, according to the formula in the code, it seems that the parameter has to be given as a relative value, not absolute as intended earlier and from the docs.

Tested MaxPowerChange with nuclear-st. Adapted for this:

expandable: False because otherwise there is no capacity and gradients cannot be calculated, apparently due to how oemof considers gradients in Flow objects at the moment (only for dispatch optimization?)
ch4-gt commented out to force nuclear-st to drive at its limits
(not pushed) Tested locally with regions AT and BE only, for faster optimization.

Also tested with gradient manually set to 0.05/h instead of 0.39/h (calculated from Scalars.csv value 3.3%/5 min) to force plants to reach  gradient limit in more timesteps (for region BE: 5144 vs. about 50 at 0.39/h – not visible!).

Added a test that reveals that there might be a bug in oemof solph.
When defining a Flow like this ...
energysystem.add(
    Source(
        label='plants',
        outputs={
            bel: Flow(
                nominal_value=None,
                investment=Investment(),
                positive_gradient={'ub': 0.2, 'costs': 0},
                negative_gradient={'ub': 0.2, 'costs': 0}
                )
        }
    )
)

... the model building raises an error:
Traceback (most recent call last):
  File "/home/unndreay/Workspaces/oemo-flex/experiment_1/scripts/FlexMex1_2a/gradient_example.py", line 63, in <module>
    optimization_model = Model(energysystem=energysystem)
  File "/home/unndreay/.virtualenvs/oemo-flex/lib/python3.7/site-packages/oemof/solph/models.py", line 267, in __init__
    super().__init__(energysystem, **kwargs)
  File "/home/unndreay/.virtualenvs/oemo-flex/lib/python3.7/site-packages/oemof/solph/models.py", line 98, in __init__
    self._construct()
  File "/home/unndreay/.virtualenvs/oemo-flex/lib/python3.7/site-packages/oemof/solph/models.py", line 105, in _construct
    self._add_child_blocks()
  File "/home/unndreay/.virtualenvs/oemo-flex/lib/python3.7/site-packages/oemof/solph/models.py", line 133, in _add_child_blocks
    block._create(group=self.es.groups.get(group))
  File "/home/unndreay/.virtualenvs/oemo-flex/lib/python3.7/site-packages/oemof/solph/blocks.py", line 138, in _create
    f.positive_gradient['ub'][t] * f.nominal_value)
TypeError: unsupported operand type(s) for *: 'float' and 'NoneType'

Nominal value is None. This error message is awkward because if you set a nominal value, there is another error telling us that is has to be set to None if an investment object is given.

After talking we decided to work around this issue by applying the gradients to the input Flow of the conversion. The actual value must take the efficiency into account. Investment remains at the output Flow.

Error persists with gradients applied to input flow. Also with disabling expansion by calling:
update_nuclear_st(exp_paths.data_preprocessed, scalars, expandable=False, from_green_field=False)

(instead of True, True)

Error persists with gradients applied to input flow. Also with disabling expansion by calling:
update_nuclear_st(exp_paths.data_preprocessed, scalars, expandable=False, from_green_field=False)

(instead of True, True)

Stop here because oemof.solph issue pending (see reference above).
Last working commit is https://github.com/modex-flexmex/oemo-flex/commit/7a77f6b76accfb49b521c06d12d535230301a89d.

Stop here because of issue upstream (see ref above).
Last working commit is https://github.com/modex-flexmex/oemo-flex/pull/44/commits/7a77f6b76accfb49b521c06d12d535230301a89d. Revert up to that commit.

Preprocessing with the current Data.zip version doesn't work:
...
19:51:02-INFO-Used oemof version: 0.3.2
19:51:02-INFO-Start running the whole modelling pipeline
Traceback (most recent call last):
  File "/home/unndreay/.virtualenvs/oemo-flex/lib/python3.7/site-packages/pandas/core/indexes/base.py", line 2646, in get_loc
    return self._engine.get_loc(key)
  File "pandas/_libs/index.pyx", line 111, in pandas._libs.index.IndexEngine.get_loc
  File "pandas/_libs/index.pyx", line 138, in pandas._libs.index.IndexEngine.get_loc
  File "pandas/_libs/hashtable_class_helper.pxi", line 1619, in pandas._libs.hashtable.PyObjectHashTable.get_item
  File "pandas/_libs/hashtable_class_helper.pxi", line 1627, in pandas._libs.hashtable.PyObjectHashTable.get_item
KeyError: 'Scenario'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/unndreay/Workspaces/oemo-flex/experiment_1/scripts/FlexMex1_2a/runall.py", line 12, in <module>
    preprocessing.main()
  File "/home/unndreay/Workspaces/oemo-flex/experiment_1/scripts/FlexMex1_2a/preprocessing.py", line 42, in main
    scalars = scalars.loc[scalars['Scenario'].isin([name, 'FlexMex1', 'ALL']), :]
  File "/home/unndreay/.virtualenvs/oemo-flex/lib/python3.7/site-packages/pandas/core/frame.py", line 2800, in __getitem__
    indexer = self.columns.get_loc(key)
  File "/home/unndreay/.virtualenvs/oemo-flex/lib/python3.7/site-packages/pandas/core/indexes/base.py", line 2648, in get_loc
    return self._engine.get_loc(self._maybe_cast_indexer(key))
  File "pandas/_libs/index.pyx", line 111, in pandas._libs.index.IndexEngine.get_loc
  File "pandas/_libs/index.pyx", line 138, in pandas._libs.index.IndexEngine.get_loc
  File "pandas/_libs/hashtable_class_helper.pxi", line 1619, in pandas._libs.hashtable.PyObjectHashTable.get_item
  File "pandas/_libs/hashtable_class_helper.pxi", line 1627, in pandas._libs.hashtable.PyObjectHashTable.get_item
KeyError: 'Scenario'

Needed to add a sep=';' to the read_csv() call for different csv format.

Same for output
...
20:00:59-INFO-Optimization successful...
20:01:12-INFO-Writing the results to /home/unndreay/Workspaces/oemo-flex/experiment_1/003_results_optimization/FlexMex1_2a
20:01:12-INFO-Restoring attributes will overwrite existing attributes.
Traceback (most recent call last):
  File "/home/unndreay/.virtualenvs/oemo-flex/lib/python3.7/site-packages/pandas/core/indexes/base.py", line 2646, in get_loc
    return self._engine.get_loc(key)
  File "pandas/_libs/index.pyx", line 111, in pandas._libs.index.IndexEngine.get_loc
  File "pandas/_libs/index.pyx", line 138, in pandas._libs.index.IndexEngine.get_loc
  File "pandas/_libs/hashtable_class_helper.pxi", line 1619, in pandas._libs.hashtable.PyObjectHashTable.get_item
  File "pandas/_libs/hashtable_class_helper.pxi", line 1627, in pandas._libs.hashtable.PyObjectHashTable.get_item
KeyError: 'Parameter'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/unndreay/Workspaces/oemo-flex/experiment_1/scripts/FlexMex1_2a/runall.py", line 18, in <module>
    postprocessing.main()
  File "/home/unndreay/Workspaces/oemo-flex/experiment_1/scripts/FlexMex1_2a/postprocessing.py", line 16, in main
    run_postprocessing(year, name, exp_paths)
  File "/home/unndreay/Workspaces/oemo-flex/oemoflex/postprocessing.py", line 568, in run_postprocessing
    fuel_cost = get_fuel_cost(carrier_cost, scalars_raw)
  File "/home/unndreay/Workspaces/oemo-flex/oemoflex/postprocessing.py", line 409, in get_fuel_cost
    price_ch4 = get_parameter_values(scalars_raw, 'Energy_Price_CH4')
  File "/home/unndreay/Workspaces/oemo-flex/oemoflex/preprocessing.py", line 206, in get_parameter_values
    is_parameter_name = scalars_df['Parameter'] == parameter_name
  File "/home/unndreay/.virtualenvs/oemo-flex/lib/python3.7/site-packages/pandas/core/frame.py", line 2800, in __getitem__
    indexer = self.columns.get_loc(key)
  File "/home/unndreay/.virtualenvs/oemo-flex/lib/python3.7/site-packages/pandas/core/indexes/base.py", line 2648, in get_loc
    return self._engine.get_loc(self._maybe_cast_indexer(key))
  File "pandas/_libs/index.pyx", line 111, in pandas._libs.index.IndexEngine.get_loc
  File "pandas/_libs/index.pyx", line 138, in pandas._libs.index.IndexEngine.get_loc
  File "pandas/_libs/hashtable_class_helper.pxi", line 1619, in pandas._libs.hashtable.PyObjectHashTable.get_item
  File "pandas/_libs/hashtable_class_helper.pxi", line 1627, in pandas._libs.hashtable.PyObjectHashTable.get_item
KeyError: 'Parameter'

The last commit is only a first try. Gives all the necessary values, but calculations are not correct. I wrote down some general thoughts on postprocessing's structure, see #60.

Finished list of missing results (const_invest and cost_fixom). Nuclear's FixOM is 0 because of a zero input!

I applied the existing structure from get_carrier_cost() to process invest_cost, fixom_cost (new) as well as emission_cost and fuel_cost (updated) depending on the technology (ch4, nuclear-st).
Emission costs in the results have decreased because it is not calculated for nuclear-st anymore.

FlexMex_Parameter_Map is a central place to map FlexMex input parameters to some universal calculations needed in postprocessing.
It could also be used for preprocessing.


Trickle Gossip Messages

LND currently constructs a new batch of  gossip messages to broadcast every 90s using a trickleTimer in the gossiper. Every time the timer expires, the messages are deduplicated and immediately broadcast to all peers that are receiving updates.
This approach creates needlessly bursty resource usage on both the sending and receiving side, and profiling confirms that this creates punctuated equilibriums in terms of bandwidth/cpu/memory. We can achieve more stable and consistent resource usage by dispersing each newly created batch over the subsequent 90s instead of delivering them all at once.
Steps to completion:

Decide how often the sub-batches should be sent, e.g. 5 seconds
Split an announcement batch into the sub-batches, where the size of each sub-batch should be roughly len(announcementBatch)/numSubBatchesIn90s, bonus points for using slice tricks
Instead of queueing the entire announcmentBatch immediately, spawn a goroutine to perform the remainder of the queueing logic for each sub-batch on the chosen interval. Note that the set of GossipSyncers can change over this period, so the syncers and filtering logic should be evaluated at the time each sub-batch is processed

Additional Considerations:

Should we enforce a minimum batch size? If we only have 10 updates to send, do we really want to space each message by 5 seconds?

Taking a stab at this

@orbitalturtle apologies, should have messaged before I started on this

@johng np :)

One problem I see with this approach is that we risk broadcasting outdated messages if an update comes in before we are able to send out the sub-batch containing it.
@cfromknecht What about adding all messages to a "delay queue", where they are required to stay for trickleDelay time before they are broadcast. This queue can take care of de-duping messages, moving them to the back of the queue if they are updated. This way we can easily rate limit how much we read from this queue, to control the flow of messages being broadcast.

@halseth i don't see that as an issue. by the time the subbatches are created, we've done all of the deduping required of us in interval i. if a newer message comes in before the batch goes out, how is it different than if we had sent the entire batch immediately and then send the newer one in the next batch?
Doing cross-interval cut through as you suggest is a good optimization, but i don't think it should prevent us from getting the simple approach in

What about adding all messages to a "delay queue", where they are required to stay for trickleDelay time before they are broadcast.

Isn't this already what we do with the trickle delay timer? Seems we'd be doing this twice?

You are right, any new update will go through the same delay, so order will be maintained ofc :)
Cutting through the new updates probably won't make that much of a difference (the biggest win would maybe be to never broadcast messages from nodes making updates with less than 90s in between)

the biggest win would maybe be to never broadcast messages from nodes making updates with less than 90s in between

I suppose that's good in some respect, though it would be at the expense of needing to buffer messages from those nodes indefinitely until they stop broadcasting as frequently :P

Closed by #2985


Qualcomm A-GPS domain - izatcloud.net

Hi. It has just been added recently in #2312 due to a misleading article. Blocking this domain will prevent A-GPS from working which means limiting phone functionality.
Here is some info:

https://www.reddit.com/r/privacy/comments/12yii9u/comment/jhojlr7/
https://blog.brixit.nl/nitrokey-dissapoints-me/

It has also been removed in hagezi/dns-blocklists#984

Thank you @hlqviet — I am removing this now.

Oh, yeah!
https://github.com/ShadowWhisperer/BlockLists/pull/95


Use acks_late=True in all tasks

Currently, all tasks receive acks even if not finished.
This PR resolves the issue involving a soft stop in celery servers.
Implementation based on https://docs.celeryproject.org/en/latest/userguide/tasks.html#Task.acks_late
When the project update celery version, can use a setting with this behavior:
https://docs.celeryproject.org/en/latest/userguide/configuration.html#std:setting-task_acks_late
I did not execute tests in my local environment, because there is not a workable environment to opensource :/

Currently, all tasks receive acks even if not finished.
This PR resolves the issue involving a soft stop in celery servers.
Implementation based on https://docs.celeryproject.org/en/latest/userguide/tasks.html#Task.acks_late
When the project update celery version, can use a setting with this behavior:
https://docs.celeryproject.org/en/latest/userguide/configuration.html#std:setting-task_acks_late
I did not execute tests in my local environment, because there is not a workable environment to opensource :/


indexer: change analyzer and payload

Uses the 'snowball' analyzer when indexing documents in order to
also respond to number strings in search queries.

Uses the grant's code field as the suggestion payload.

Signed-off-by: Alexander Ioannidis<EMAIL_ADDRESS>

The first problem is that code is not unique (only internal id is) and the second problem is that of backward compatibility. The old REST API only supported FP7, and wasn't forward looking enough to take into account other funders. So to adding legacy_id (which is the unique grant id that the legacy API will take), allow you to not apply any logic. All ids comes from just the legacy id.
Because we also don't want to be bound to above problem, then we keep the id as is.
Btw, you'll need to rebase.


scheduler: fix elastic quota TestPlugin_OnPodDelete random panic

Ⅰ. Describe what this PR does

Ⅱ. Does this pull request fix one issue?

Ⅲ. Describe how to verify it
Ⅳ. Special notes for reviews
V. Checklist

[ ] I have written necessary docs and comments
[ ] I have added necessary unit tests and integration tests
[ ] All checks passed in make test

/lgtm

/approve


New Extension: HS Utilities

I think there should be an extension which adds 6 new blocks.

This is my thumbnail idea.

I hope you like this idea.
Note: I'm new to Github.

Source code: https://github.com/horizontal-shading/HS-Extensions

Many of these blocks are in other extensions, this is probably a no.

I think there should be an extension which adds 6 new blocks.

This is my thumbnail idea.

I hope you like this idea.
Note: I'm new to Github.

Random Chinese letter? Really?

I'm closing this issue

This comment was marked as off-topic.


fix: Should properly look for element when removing

This fixes https://github.com/flame-engine/flame/issues/2973

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

2 of 2 (100.0%)  changed or added relevant lines in 1 file are covered.
1 unchanged line in 1 file lost coverage.
Overall coverage remained the same at 96.842%

Files with Coverage Reduction
New Missed Lines
%

lib/ordered_set.dart
1
96.23%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
92

Relevant Lines:
95

💛  - Coveralls


Make .str/.dt available for Series of type category with string/datetime

If a series is a type category and the underlying Categorical has categories of type string or datetime, then make it possible to use the .str/.dt assessor on such a series.
The string/dt methods work on the categories (and therefore fast if we have
only a few categories), but return a Series with a dtype other than
category (integer, boolean, string,...), so that it is no different if we use
.str / .dt on a series of type string or of type category.
The main reason for that is that I think things like s.str.slice(...) + s.str.slice(...) should work.
Closes: https://github.com/pydata/pandas/issues/10661

I've no idea why .cat is not in dir(series_of_type_category)?!?

@JanSchulz
https://github.com/pydata/pandas/blob/master/pandas/core/series.py#L2675

@kawochen Thanks! 7a20c36

Yay, it's green :-)

can you update the doc as well (maybe a reference / example in the Categorical section to one in the TextMethods section)

Added docs in categorical.rst

Ok, @jreback, you were right, it needed more tests...
Rebased + new tests + fixes... Lets wait what travis says...

Ok, two failures: https://travis-ci.org/pydata/pandas/builds/91286698
I can't reproduce both errors (in py3.5 and py2.7, and it seems I'm too stupid to get a proper py3.4 environment which can run python setup.py build_ext --inplace :-( )
On py3.3/3.4:
nosetests pandas.io.tests.test_data
=====================================================================
ERROR: test_get_underlying_price (pandas.io.tests.test_data.TestYahooOptions)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/build/pydata/pandas/pandas/util/testing.py", line 1774, in wrapper
    return t(*args, **kwargs)
  File "/home/travis/build/pydata/pandas/pandas/io/tests/test_data.py", line 400, in test_get_underlying_price
    quote_price = options_object._underlying_price_from_root(root)
  File "/home/travis/build/pydata/pandas/pandas/io/data.py", line 737, in _underlying_price_from_root
    underlying_price = root.xpath('.//*[@class="time_rtq_ticker Fz-30 Fw-b"]')[0]\
IndexError: list index out of range

Py27, https://travis-ci.org/pydata/pandas/jobs/91286713
nosetests pandas.tseries.tests.test_timeseries
======================================================================
FAIL: test_datetime64_with_DateOffset (pandas.tseries.tests.test_timeseries.TestDatetimeIndex)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/build/pydata/pandas/pandas/tseries/tests/test_timeseries.py", line 2668, in test_datetime64_with_DateOffset
    assert_func(klass([op + x for x in s]), op + s)
  File "/opt/python/2.7.9/lib/python2.7/contextlib.py", line 24, in __exit__
    self.gen.next()
  File "/home/travis/build/pydata/pandas/pandas/util/testing.py", line 2042, in assert_produces_warning
    % expected_warning.__name__)
AssertionError: Did not see expected warning of class 'PerformanceWarning'.

ignore the first one, 2nd I am fixing now.

Ok, remodeled the tests and rebased. You could cancel the following travis run, the differnece to the latest is just some reshuffeling in the commits: https://travis-ci.org/pydata/pandas/builds/91400953

Yay, https://travis-ci.org/pydata/pandas/builds/91400953 is green so hopefull this one (no code change in the PR, only moved some changes to a different commit) will also be gree :-)

@jreback green... :-) Thanks for fixing whatever needed fixing :-)

@jreback Hopefully addressed your comments. Thanks for the review!

@JanSchulz Nice! I just added a few minor doc comments

@jorisvandenbossche Addressed your comments. Thanks for the review!

couple comments. pls squash when finished.

@jreback: please no squash: I really put some work into making each commit stand on its own, the unit of change is each commit (4 of them...) and not the complete PR.

Ok, addressed the comments, please review the doc change if that is clearer now.

ok, lgtm. ping on green.
on the squash; ok for this one :) normally we just squashem as a matter of course.

@JanSchulz ok, ping when green (after any more doc changes you have)

Ok, added something to text.rst. Please review. It would alos good if someone with access to travis would cancel the other builds (https://travis-ci.org/pydata/pandas/builds/91604782 and https://travis-ci.org/pydata/pandas/builds/91605534)  so that we have 4 hours less to wait for this :-)

@jreback ping :-)

merged via https://github.com/pydata/pandas/commit/6eefe756a9cdd8d8b9efc95920f8434245bba8cf
thanks!
as usual, pls review built docs for accuracy

@jreback Thanks for all the reviews and for your patience! The PR got much better thanks to you and @jorisvandenbossche!
Will add a PR forthis:
In [124]: date_s = pd.Series(date_range('1/1/2015', periods=5))
---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
<ipython-input-124-f7c79f210480> in <module>()
----> 1 date_s = pd.Series(date_range('1/1/2015', periods=5))

NameError: name 'date_range' is not defined

@JanSchulz no thank you!
great PR as always from you.
I got the doc fix, was making some corrections anyhow.


Documentation Error

@harshmahesheka can you sign the commit ?

Yeah, sure.

@harshmahesheka can you sign the commit ?
Yeah,sure

This pull request has been mentioned on Gazebo Community. There might be relevant details there:
https://community.gazebosim.org/t/new-ignition-releases-2022-03-01-citadel-edifice-fortress/1313/1

This pull request has been mentioned on Gazebo Community. There might be relevant details there:
https://community.gazebosim.org/t/new-ignition-releases-2022-04-13-fortress-edifice/1367/1


KDtree type mismatch in knnW

From pysal-dev:

When I am running the following code for weight generation
import pysal
import numpy as np
x,y = np.indices((5,5))
x.shape = y.shape = (25,1)
data = np.hstack([x,y])
wknn3 = pysal.knnW(data, k=3)
wknn3.neighbors[0]
wknn.s0

then I am getting the following error:

Traceback (most recent call last):
File "C:\Python27\FOSS4G NA 2015 - Spatial Data Analysis in Python\PySAL_Workshop\mine\knnweight.py", line 9, in
wknn3 = pysal.knnW(data, k = 3)
File "C:\Python27\FOSS4G NA 2015 - Spatial Data Analysis in Python\PySAL_Workshop\mine\pysal\weights\Distance.py", line 86, in knnW
nnq = kdtree.query(data, k=k+1, p=p)
AttributeError: 'numpy.ndarray' object has no attribute 'query'
> Why? Please help!

The error actually occurs right at the knnW weights constructor.
It looks like commit a38c7d2c removed dispatch on type of input for the knnW constructor.
It used to accept numpy arrays and silently build the KDTree behind the scenes. Now, it will fail immediately if the user passes a previously-valid input.
Potential issues with this practice.

We never issued any deprecation warnings on this change.
This thumps a 2015 FOSS4G workshop's code
Previously, proccessing ndarrays reasonably as the source for what the user intends is a KDTree obeys Postel's Law, but definitely includes unnecessary chrome around the core algorithm. (this is where I like multiple dispatch :)

But, I see nothing wrong with @jlaura's commit itself. It's just an API change that was not smoothed by a deprecation warning.
I'd mark this as notabug+wontfix, but perspective is needed.

The OP's code was copy/pasted from the user guide: http://pysal.readthedocs.org/en/latest/users/tutorials/weights.html#k-nearest-neighbor-weights
(Larger issue, are the examples the user guide no longer tested?)

Did they used to get tested? I'm unaware of any tests that run aside from the unittests. I think even doctests are turned off.

Further complicating things, we do a checks in every constructor in Distance.py for whether something is already a KDTree. Commit b39a43db replaced all of our default constructors with cKDTrees, rather than KDTrees.
While it's reasonable to assume that cKDTree <: KDTree, it's not the case:
1]> import scipy.spatial as spat; import pysal as ps

2]> df = ps.pdio.read_files(ps.examples.get_path('columbus.shp'))

3]> data = df[['X', 'Y']].values

4]> print(type(ps.cg.KDTree(data)))
4|   ckdtree.cKDTree

5]> issubclass(type(spat.cKDTree), spat.KDTree)
5|   False

6]> issubclass(type(spat.KDTree), spat.cKDTree)
6|   False

which means that we've now got this class of errors:
ps.Kernel(ps.cg.KDTree(data))
TypeError                                 Traceback (most recent call last)
<ipython-input-13-8e6cf2c300ce> in <module>()
----> 1 ps.Kernel(ps.cg.KDTree(data))

/home/ljw/.local/lib/python2.7/site-packages/pysal/weights/Distance.pyc in __init__(self, data, bandwidth, fixed, k, function, eps, ids, diagonal)
    271         else:
    272             self.data = data
--> 273             self.kdt = KDTree(self.data)
    274         self.k = k + 1
    275         self.function = function.lower()

/home/ljw/.local/lib/python2.7/site-packages/pysal/cg/kdtree.pyc in KDTree(data, leafsize, distance_metric, radius)
    242             return scipy.spatial.KDTree(data, leafsize)
    243         else:
--> 244             return scipy.spatial.cKDTree(data, leafsize)
    245     elif distance_metric == 'Arc':
    246         return Arc_KDTree(data, leafsize, radius)

ckdtree.pyx in ckdtree.cKDTree.__init__ (scipy/spatial/ckdtree/ckdtree.cxx:7474)()

/home/ljw/.local/lib/python2.7/site-packages/numpy/core/numeric.pyc in ascontiguousarray(a, dtype)
    559
    560     """
--> 561     return array(a, dtype, copy=False, order='C', ndmin=1)
    562
    563 def asfortranarray(a, dtype=None):

TypeError: float() argument must be a string or a number

AFAICT, this hits every constructor in Distance.py that dispatches on the input type. Fortunately, the fix is clear: add cKDTree to the set of kdtree types we check in the input.
Agian, the main issue here is that the inheritance diagram is forked: pysal.cg.Arc_KDTree and pysal.cg.KDTree have no common parent type beyond object.

At some point the tests in the documentation were turned off.

This was resolved in #786 I believe.

Indeed:
1]> import pysal as ps, numpy as np
2]> x,y = np.indices((5,5))
3]> x.shape = y.shape = (25,1)
4]> data = np.hstack((x,y))
5]> wknn3 = ps.knnW(data, k=3)
6]> ps.version
6|  '1.11.2dev'

proceeds without a hitch.


Allow upload package to S3 using the indicated role

Looks like Zappa uploads the artifact to S3 as current user instead of using the defined role?
With zappa_settings.json:
{
    "production": {
        "app_function": "linkportal.app",
        "s3_bucket": "musttu-zappa",
        "manage_roles": false,
        "role_name": "my-custom-role"
    }
}

14:09 $ zappa deploy production
Packaging project as zip...
Warning! Your project and virtualenv have the same name! You may want to re-create your venv with a new name, or explicitly define a 'project_name', as this may cause errors.
Uploading linkportal-production-1476788948.zip (7.2MiB)...
  0%|           | 0.00/7.54M [00:00<?, ?B/s]
Failed to upload linkportal-production-1476788948.zip to musttu-zappa/linkportal-production-1476788948.zip: An error occurred (AccessDenied) when calling the PutObject operation: Access Denied
Unable to upload to S3. Quitting.

In our corporation, we users have only read-only permissions and we use assume-role to obtain more permissions when we need them.

The role_name is for the execution role of the remote Zappa function, the local role should be defined your aws credentials.

https://github.com/Miserlou/Zappa#using-custom-aws-iam-roles-and-policies

@Miserlou Thanks for the pointers. I am a bit lost here, though:

the local role should be defined your aws credentials.

By "local role", do you mean the roles/permissions bound to my user default authorization (determined as [default] in ~/.aws or as AWS_* env variables)? That is exactly the problem here - by default I only have read-only access to AWS. I guess I need to assume other role (with write access to S3 bucket), but in that case I need to know what permissions I need to attach to that role? At least uploading to S3 bucket...?
Also, could you clarify what are attach_policy and assume_policy files in config? If these are specified, will Zappa create them or what will actually happen... or, I didn't think/know that you can assume policies on-the-fly? Or, are attach_policy and assume_policy what is attached to created role if manage_roles is enabled?

Yeah, your local AWS credentials file. It assumes default by default, but you can specify a different one with "profile_name": "your-profile-name", in your zappa_settings.
You will always need to be able to upload to an S3 bucket to use Zappa, so I think creating the attaching the ability to do that for the local user is outside the scope of Zappa itself, you should simply have an account that has the ability to upload to a location in your code bucket, I don't see why that needs to be dynamic.
Re: the remote policy, discussion is still happening here: https://github.com/Miserlou/Zappa/issues/244
If manage_roles is enabled, it will create them, either from the default, or from the user-supplied templates in those settings. That's all for remote settings though.
Make sense? Am I missing something?

Also possibly relevant: https://github.com/bjinwright/zops

Make sense? Am I missing something?

Yeah, it makes sense. So "local role" (=the default role/user/principal used) must have permissions listed in https://github.com/Miserlou/Zappa/issues/244#issuecomment-241874091.
And zappa role (role_name) must have policies presented in https://github.com/Miserlou/Zappa/issues/244#issuecomment-239499405. Those are the defaults, so there shouldn't be need to define custom attach_policy and assume_policy. Correct me if wrong.
I guess I got confused by this in bit in readme:

By default, the Zappa client will create and manage the necessary IAM policies and roles to DEPLOY and execute Zappa applications.

While actually Zappa will only create the role/policies needed to execute/run the applications. Deploying is done using credentials that you have active (profile_name).
Did I get it right now?

Yes. I will update the documentation to clarify that. Sorry for the confusion.

Alright, I think we are done here. Thanks for prompt support!


Optimized autoraster

Initial commit for optimized autoraster, work in progress
Do the ophyd devices defined need to be in mxtools or stay here?
What is the criteria of having devices in mxtools? As there are multiple devices in start_bs.py and setenergy_lsdc.py
In this PR there are 2 major contributions:

A bluesky plan to align the pin on mount using the top view camera as part of state transitions from SE -> TA -> SA
A bluesky plan to replace autoRasterLoop that uses an ML model to identify a bounding box around the sample and run 2 2D rasters around the face on and orthogonal positions

I can't review this very thoroughly, but there are various comments and changes that need to be addressed here.

Understandable, I'm in the process of addressing issues with the bluesky plan itself (error handling, correctness of motions etc.). Then I will address your initial comments before I ask for a more detailed review

Thorough testing of code done at both beamlines. PR ready for full review

My main concern is whether this pull request has code that is maintainable and extensible in the long-term. This may indicate that we should do an architecture review to understand how to enable new methods such as this and the MD2 collections on NYX.

As far as I understand it this code is pretty modular, devices and plans related to this are in their own python modules. Options are available via the GUI to select whether to use this feature or not. start_bs.py specifically initializes devices for each beamline. The 2 plans developed as part of this PR has only one way to use it daq_macros.run_top_view_optimized() and daq_macros.run_loop_center_plan() both of which can be gated behind an if beamline == 'amx' or beamline == fmx` statement. Or add conditions within that one function if nyx wants to do things their own way.
Are there specific long term concerns you have with this code? I can attempt to address them in a future PR

I think I went through a picky phase when doing the review and should have kept in mind: "premature optimization". I think you will see over time where the parts that need more flexibility due to changing parameters, and changing hardware/etc, so you should refactor then.


Ensure successful storage of cryto materials in vault by Indy helm charts

Description
As a developer I want to check and ensure that the store certs are successfully storing the cryto materials in vault.
Acceptance Criteria

Checks to ensure cryto successfully stored
Retry logic based on the above check
Add retry logic for reading from vault

I will work on this issue


An option to hide config credentials

Instances of SHConfig are usually passed around in the code a lot, which means that some frameworks, such as eo-learn, would log it and write into a report. But the problem is that a representation of a config object reveals all its parameters, including its credentials.
This PR adds an option to hide credentials. It also refactors unit tests for SHConfig and adds a few new ones.

Codecov Report

Merging #191 (e0569d2) into develop (d3b7420) will increase coverage by 0.11%.
The diff coverage is 80.95%.

@@             Coverage Diff             @@
##           develop     #191      +/-   ##
===========================================
+ Coverage    89.66%   89.77%   +0.11%
===========================================
  Files           38       38
  Lines         3695     3707      +12
===========================================
+ Hits          3313     3328      +15
+ Misses         382      379       -3

Impacted Files
Coverage Δ

sentinelhub/config.py
90.16% <80.95%> (+3.80%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d3b7420...e0569d2. Read the comment docs.


Voice kit 'night mode' for LED

Is there any way of setting a lower LED peak brightness during 'night' hours?
Before I hack away at src/aiy/_drivers/_led.py,  I was wondering if there was an existing option.  If not, is src/aiy/_drivers/_led.py the best place to implement such a feature?
I was thinking that it should be possible to replace all hard-coded peak brightnesses of 100 to a variable which can be changed to reflect the 'mode'.  Am I wrong?

It might be easier to just change the line:
self.pwm.ChangeDutyCycle(next(self.iterator))
to multiply by the max brightness / 100. Then the main script can call set_max_brightness(50) at night.

Does such a function already exist?  If not, is there a preferred placement for such a function?  Would the voicehat object be the best place to attach these variables / functions?

My custom service script already supports the use of a settings file to store various details (podcast URLs, RF codes for remote devices, credentials for other web services).  I can easily store the settings there, but I would require a way of passing these variables to the underlying objects in a way that would not break any coding bible that this project adheres to.

I would add a new set_max_brightness() method to the LED class, which can optionally be used to adapt the brightness. The user's main.py can then set the max brightness however it wants.

After taking the advice from @drigz, I tried the following:
I added this to the init function of the led driver:
self.brightness = 1

I then added the following function:
    def set_brightness(self, brightness):
        try:
            self.brightness = int(brightness) / 100
        except ValueError:
            raise ValueError('unsupported brightness: %s' % brightness)

The last change was to modify every call to self.pwm.ChangeDutyCycle as follows:
self.pwm.ChangeDutyCycle(0 *** self.brightness**)

or
self.pwm.ChangeDutyCycle(100 *** self.brightness**)

or
self.pwm.ChangeDutyCycle(next(self.iterator) * self.brightness)

Now, I can set the brightness by simply calling
aiy.voicehat.get_led().set_brightness(10)

Is there anything I have overlooked?
Since I'm not very familiar with git, I don't know how to generate a pull request.

I just submitted a PR to add this functionality. Also wanted to make a note that although 10% seems like a low value to use, it actually corresponds to around 38% of the LED's full brightness according to the equation for linear LED PWM found here.

Since originally submitting this code, I have amended it.  The latest version can be found here:
https://github.com/mpember/aiyprojects-raspbian/blob/aiyprojects/src/aiy/_drivers/_led.py
I have two AIY kits.  Both came with the green button.  Since I wanted to use a different colour on one of the kits, I replaced the button an arcade button I bought in a kit of 10.  The kit that contained the original button suffered from problem of spiking to full brightness at random points in the cycle.  I did not have this issue with the kit that included a replacement button.
Since switching to the new method of adjusting the brightness, the problem appears to have gone away.
I'm not sure if this new approach makes it possible to better correlate the brightness percentage with visual brightness.

I recently bought an AIY kit and am using it with the original green button. The light is much brighter than the tiny LED I was using before which is why I wanted night mode.
I've been using the code in my PR for the past few days and haven't noticed any weird behavior with the light. In theory the two approaches should behave exactly the same, they're just multiplying by self.brightness at different places in the code.
I know that in practice things don't always work as theoretically expected, so there could be a problem with the current approach that I'm not aware of. However the correlation of the brightness percentage with visual brightness is definitely the same between the two approaches.

I guess my second approach was a bit of overkill.  If others are not having the same problem that I experienced, that's great.  I've got a few more of those replacement buttons.  I should probably try another LED/button and see if it just happens to be a dodgy bit of hardware in that specific kit.  I guess the only real improvement is that my second approach kept the maximum brightness for the ON status.
While I have modified the value of BEACON_DARK to be 0 as a way of stopping the idle flashing, I wonder if a 'night mode' would also involve stopping the AIY kit from acting as an accidental night light.


chore: bdd-spring-1573037117 to 0.0.1

chore: Promote bdd-spring-1573037117 to version 0.0.1

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by:
To fully approve this pull request, please assign additional approvers.
We suggest the following additional approvers:
If they are not already assigned, you can assign the PR to them by writing /assign  in a comment when ready.
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment


Added Weather CLI scripts (new script)

Weather CLI
Weather_cli
Created a new script for retrieving weather data from cli using api.openweathermap.org
Self Check(Tick After Making pull Request)

[ ] This issue was assigned to me.
[x] One Change in one Pull Request
[x] My file is in proper folder
[x] I am following clean code and Documentation and my code is well linted with flake8.
[x] I have added README.md and requirements.txt with my script

If issue was not assigned to you Please don't make a PR. It will marked as invalid.
Join Us on Discord:- https://discord.gg/JfbK3bS
acknowledgement

[x] I have tested the script and distributing it under MIT license.

Thanks for making the pull request. but there is already the same script exist in the repo.
https://github.com/Py-Contributors/awesomeScripts/tree/master/check_weather
If you like to contribute I will suggest you the current script.
Thanks

I checked the script before i make the PR. My script uses the same API but it's a command line with more arguments (and more will be added soon). Sorry for redundancy.

Thanks for contributing to the repository. Looking more contribution in future.
Join Py-Contributors Community
Discord  • Telegram  • website
Please share your Hacktober Experience on Discord Channel to Encourage more member to contribute to open source.
Happy Coding


Fix: piracy/activation.mspx was a 404 page

Sorry, I noticed something off after #950 got merged. Unfortunately the diffs with long text are odd to review, but it looks like I had copy/pasted the link and fixed the pasted version instead of the original.
At the same time, I found that the old link https://www.microsoft.com/piracy/activation.mspx was dead , but removing the .mspx redirected to https://www.microsoft.com/en-us/piracy/mpa.aspx

@nschonni : Thanks for your contribution! The author, @evanwindom, has been notified to review your proposed change.

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

subscriptions/vs-license-terms.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

subscriptions/vs-license-terms.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

@nschonni : Thanks for your contribution! The author, @evanwindom, has been notified to review your proposed change.