fix(entity-store): improve type safety of entity store upsert operations

Explicitly require an initialization callback if entities do not already exist.
Fix #436
PR Checklist
Please check if your PR fulfills the following requirements:

[x] The commit message follows our guidelines: https://github.com/datorama/akita/blob/master/CONTRIBUTING.md#commit
[x] Tests for the changes have been added (for bug fixes / features)
[x] Docs have been added / updated (for bug fixes / features)

PR Type
What kind of change does this PR introduce?

[x] Bugfix
[ ] Feature
[ ] Code style update (formatting, local variables)
[ ] Refactoring (no functional changes, no api changes)
[ ] Build related changes
[ ] CI related changes
[ ] Documentation content changes
[ ] Other... Please describe:

What is the current behavior?
Issue Number: #436
Does this PR introduce a breaking change?

[x] Yes
[ ] No

The function signature of upsert in EntityStore has changed. As a side effect, runStoreAction has been revised to support the new upsert function: The previous function overloading of runStoreAction caused several problems with the new upsert signature and was replaced by a discriminated action config object.

We'll need to bump a major version to add this feature. There is no way to do it without introducing a breaking change?

I added backward compatibility to the upsert function but the runStoreActions are still a breaking change. The runStoreActions itself support the deprecated upsert call, but the action instantiation is still different to the original implementation.

Can you summarize the changes, please?

A short summary:

Refactored upsert function to safely update or insert entities by an optional initialization callback:

new function signature with a onCreate callback for initialization if entity is previously not existing.
marked old function signature  as deprecated because it violates type safety of the entity store.

Rewrite libs/akita/src/lib/runStoreAction.ts:

support type safe entity upsert actions.
refactored upsertMany into its own action: StoreActions.UpsertManyEntities().
avoid type conflicts issued by the previous function overloading of runStoreAction.

Adjusted docs and tests

Two things:

Do we have breaking changes?
Quality Check / Build applications & libraries (pull_request) - is failing.

Yep, runStoreAction is the only breaking change. The new upsert function in EntityStore is fully backwards compatible.
The failed job is fixed: ngc seems not to like this references in a static context.

Yep, runStoreAction is the only breaking change

Can we prevent it?

Maybe, with a lot of conditional type generics in the runStoreAction definition (was my first attempt; and which make it really difficult to maintain), but I have not found any suitable solution. So I switched to a params object wich uses the action type as discriminant, so typescript itself can infer the appropriate type. It will therefore amount to a breaking change.

If we bump to v5, I prefer to remove deprecations and keep the code clean. Please do it.
And please write in the https://github.com/datorama/akita/blob/master/BREAKING_CHANGES.md file the changes with before and after examples.
Thanks.

I updated BREAKING_CHANGES and removed deprecated functions.
I also rewrite runStoreAction a second time because there were still type check issues with the previous implementation. The idea know is to specify the store class, the operation to perform and execute it through a callback with the appropriate operator arguments. Please verify that the way runStoreAction works now is ok for you.

Please resolve the conflicts.

I think that some of your changes cause the build to fail. Can you check it, please?
https://github.com/datorama/akita/runs/769598715?check_suite_focus=true

I want to release the new version.

The compiler also throws this error with previous commits. It seems that Babel has a problem with node v12/v13, with node v10 its building without errors: https://github.com/nodejs/node/issues/32852#issuecomment-613655150

Did you change the node version?

Nope, I don't have an explanation why it's failing now. But I have developed and tested locally against node v10.

Try to downgrade to node 10, and test it.

I can also verify this error for previous commits (e.g. I checked<EMAIL_ADDRESS>https://github.com/datorama/akita/commit/76e383f3712cc36af2f7936132c273e52836a80e, before mine).

Thanks

This broke a tonne of my code with errors like:
 Property 'data' does not exist on type '{} | OrderSummaryReport'.

For example this is the pattern I was using:
 this.userSessionStore.upsert(id, state => {

                if (state.savepoints)
                {
                        // add new savepoint to list
                        return {
                            ...state,
                            savepoints: [...state.savepoints, newSavepointData]
                        };
                    }
                }
                else
                {
                    // create new session
                    return {
                        savepoints: [data]
                    };
                }
            });

Now it complains that savePoints doesn't exist on {} | UserSession
Just wanted to mention that for me it was a breaking change.
I can change it to if ('savepoints' in state) { ... } then it works again.
Or maybe I'm completely using the upsert method incorrectly!

@simeyla Sorry to hear that. This was introduced in v5 as a breaking change. This behaviour is intentional, because if the entity does not exist before, an empty object is passed. The problem is, if a state consists of nested properties, you have to check wether this object is empty (non existing) or already exists with all required properties properly initialized. E.g.:
interface UserSession {
  a: {
     b: number
  }
}

Then this would throw an error if the entity does not exist, because the state argument is {} and an undefined property a is accessed:
 this.userSessionStore.upsert(id, state => ({ a: { b: state.a.b++  }}))

So as you mentioned you have to use a type guard to check for the presence of the state.

@xbaun shame you can't just say == {}

Try this type guard:
function isUpdate<T>(a: any): a is T {
    return !(Object.keys(a).length === 0 && Object.getPrototypeOf(a).constructor === Object.prototype.constructor);
}

Playground Link


Resurrection of Huperziaceae (Lycopodiales)

Author(s) of proposal
Li-Bing Zhang, Xin-Mao Zhou
Name of taxon
Huperziaceae
Rank of taxon
Family
Approximate number of species affected
280
Description of change
Recognition of two families in Lycopodiales: Huperziaceae and Lycopodiaceae s.s. This resurrection won't change any names of species or genera.
Reason for change
We advocate for the recognition of Huperziaceae for three reasons: morphological diagnosability, spore ornamentation, and deep divergence between Huperziaceae and Lycopodiaceae s.s.
(1) There are a number of morphological characters defining Huperziaceae (Zhang & Iwatsuki, 2013; Øllgaard & Windisch, 2016; Chen et al., 2022; Zhang & Zhou, 2022). Notably, the spore ornamentations in Huperziaceae are exclusively foveolate-fossulate, whereas those in Lycopodiaceae are reticulate, scabrate, baculate, rugulate, or nearly reticulate (Chen et al., 2022; Zhang & Zhou, 2022).
(2) Ancient divergence: Testo et al. (2018) estimated a divergence time between Huperziaceae and Lycopodiaceae s.s. as 368.4 million years ago (mya) vs. 377.8 mya between Isoëtales and Selaginellales; or as 308.8 mya vs. 362.7 mya based on the uncorrelated lognormal clock model.
(3) Recognizing Huperziaceae is not new and a number of botanists favored two families in Lycopodiales [Rothmaler, 1944, 1962 (as Urostachyaceae); Ching, 1978, 1983; Holub, 1985; Zhang & Kung, 2000; Haines, 2003; Zhang, 2004].
Huperziaceae contain three genera: Huperzia with ca. 30 spp., Phlegmariurus with ca. 250 spp., and Phylloglossum with 1 sp.
Reference(s) for publication of the name
Haines, 2003_ The families Huperziaceae and Lycopodiaceae in New England
Zhang & Iwatsuki, 2013, Flora of China.
Ollgaard & Windish 2016_Brazil Lycopodiaceae.pdf
Testo et al. 2018_Lycopodiaceae biogeography.pdf
Chen et al 2022_Lycopodiaceae.pdf
Zhang & Zhou 2022_classification of Lycopodiales.pdf
List the numbers of any related issues
No response
Code of Conduct

[X] I agree to follow the PPG Code of Conduct

Hi Li-Bing,
Thanks for this proposal - I'll take this opportunity to weigh in. While this is certainly a monophyletic group, I am not in favor of recognizing it as a family, mostly because I think that Lycopodiaceae in the sense used by most authors (and by PPG 1) is much more readily diagnosable morphologically. I think that this is the reason that most students of the group, have not recognized Huperziaceae. This includes all of the publications you cite as references, except for Arthur Haines' treatment for New England, USA.
Lycopodiaceae in the commonly used circumscription can be easily recognized - they are homosporous plants with eligulate microphylls, trilete spores, and protosteles. Based on my understanding of the morphology of the plants and the results of the papers that you cite, Huperziaceae is characterized morphologically principally by spore ornamentation. The inclusion of Phylloglossum in this clade makes Huperziaceae rather morphologically heterogeneous and difficult to characterize, n my opinion.
I also worry that applying the criterion of divergence times here is fraught - drawing from the Testo et al. 2018 paper that is cited here, all or nearly all genera of Lycopodiaceae (sensu PPG 1) diverged prior to the K-T boundary. Given that these genera are often roughly as old as many fern genera and are probably more readily morphologically diagnosable than Huperziaceae/Lycopodiaceae s.s., one could just as readily push to recognize these all as families.
Finally, I don't know if the fact that a name has already been published is a compelling reason for us to take it back up here. While there are several publication that have adopted the use of Huperziaceae since it was first published by Rothmaler, these are a rather small minority. Most taxonomists working these plants have chosen to recognize this clade at the subfamily level.
For the same reasons of morphological diagnosability and concerns about using clade age as a criterion for rank assignment, I prefer to not recognize Lycopodiastroideae. I would also be opposed to the recognition of Lycopodielloideae, though I would find that would be an easier clade to diagnose on the basis of morphology than Huperziaceae would be.
Best,
Wes

Hi Li-Bing,
I agree with Wes, my preference is for a single family Lycopodiaceae, keeping Huperzioideae as a subfamily. Two reasons for my preferance are:
(1) Lycopodiaceae is readily diagnosable in the field based on numerous macro-morphological features and hand-lense features. Huperzioideae is diagnosable on microscopic features. There are macro-morphological features that ALMOST work (tufted branches, tufted epicorticular roots, isodichotomy) but there are so many exceptions in Phlegmariurus and the exceptional Phylloglossum.
(2) Lycopodiaceae has a long and stable popular-science history as an educational tool. It is often the only thing a user knows about these plants, and they can use that key to access more knowledge.
Where a taxon is already monophyletic, diagnosable, stable, popular and still usefull in its current form, I prefer retaining those benefits.
Regards, Ashley

Hi Wes, Ashley and Edgar, thanks for your comments. In the six references I cited, In addition to Arthur Haines' treatment, Zhang & Zhou (2022) also adopted Huperziaceae. I am sure you know a lot of references in which Huperziaceae (= Urostachyaceae) is constantly adopted, e.g., Rothmaler (1944), Ching (1978, 1981a, b, 1982), Zhang & Kung (1998, 1999, 2000, Zhang (2004; Flora of China), Ji et al. (2008), Hsieh (2012), Xu et al. (2017), Weakley (2020), Weakley & Southeastern Flora Team (2022, 2023, 2024), quite a few US state floras, ... and quite a few online sites including Michael Sundue's Ferns of the World: https://www.fernsoftheworld.com/fow_family/huperziaceae/
Yes, the broadly defined Lycopodiaceae is dominant because (1) Lycopodiaceae is an older name and plants of Lycopodiaceae s.s. are more common; and (2) the literature in the past 40+ years has been influenced mainly by our dear colleague, Prof. B. Øllgaard, who adopted Lycopodiaceae s.l. We know Polypodiaceae s.l. had been dominant too, but we now recognize 20+ families segregated from Polypodiaceae s.l.
No, spore ornamentation is not the only feature defining the two families. What distinguishes the two current subfamilies, Huperzioideae and Lycopodioideae, distinguishes the two families too. For example, horizontal stems absent, shoots clustered, roots usually in single basal clump in H. vs. horizontal stems present, shoots non-clustered, roots emerging at intervals along horizontal stem in L.
Well, I see your point about divergence times. On the other hand, divergence time is not the only criterium we use when deciding a taxonomic rank, right?
Similar arguments can be applied to the recognition of Lycopodiastroideae: morphology, phylogeny, and ancient divergence.
Best wishes, Li-Bing

Great discussion everyone.
Since my name and ferns of the world was mentioned, I should point out that
the content of FOTW posts belong to the authors of that post, not me. They
reflect the taxonomic choice of the authors at that time and should not be
considered a formal classification or one that I promote or impose.  I do
spend a lot of time developing the site, but the content really belongs to
the contributors.
By the way, if anyone wants to submit posts, please get in touch! Direct
message me at @.*** and I will get you started.
Best,
Michael
On Tue, Apr 2, 2024 at 8:27 PM Li-Bing Zhang @.***>
wrote:

Hi Wes, Ashley and Edgar, thanks for your comments. In the six references
I cited, In addition to Arthur Haines' treatment, Zhang & Zhou (2022) also
adopted Huperziaceae. I am sure you know a lot of references in which
Huperziaceae (= Urostachyaceae) is constantly adopted, e.g., Rothmaler
(1944), Ching (1978, 1981a, b, 1982), Zhang & Kung (1998, 1999, 2000, Zhang
(2004; Flora of China), Ji et al. (2008), Hsieh (2012), Xu et al. (2017),
Weakley (2020), Weakley & Southeastern Flora Team (2022, 2023, 2024), quite
a few US state floras, ... and quite a few online sites including Michael
Sundue's Ferns of the World:
https://www.fernsoftheworld.com/fow_family/huperziaceae/
Yes, the broadly defined Lycopodiaceae is dominant because (1)
Lycopodiaceae is an older name and plants of Lycopodiaceae s.s. are more
common; and (2) the literature in the past 40+ years has been influenced
mainly by our dear colleague, Prof. B. Øllgaard, who adopted Lycopodiaceae
s.l. We know Polypodiaceae s.l. had been dominant too, but we now recognize
20+ families segregated from Polypodiaceae s.l.
No, spore ornamentation is not the only feature defining the two families.
What distinguishes the two current subfamilies, Huperzioideae and
Lycopodioideae, distinguishes the two families too. For example, horizontal
stems absent, shoots clustered, roots usually in single basal clump in H.
vs. horizontal stems present, shoots non-clustered, roots emerging at
intervals along horizontal stem in L.
Well, I see your point about divergence times. On the other hand,
divergence time is not the only criterium we use when deciding a taxonomic
rank, right?
Similar arguments can be applied to the recognition of Lycopodiastroideae:
morphology, phylogeny, and ancient divergence.
Best wishes, Li-Bing
—
Reply to this email directly, view it on GitHub
https://github.com/pteridogroup/ppg/issues/75#issuecomment-2032943480,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABXVFQXUPLFIIAHOTGGUGFTY3MBDPAVCNFSM6AAAAABDHXVJKKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMZSHE2DGNBYGA
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

--
Michael Sundue
Fernsoftheworld.com

Hi Michael, thanks for clarification :-).
Hi Ashley, I would not worry too much about some exceptions, as long as it works in most cases. We have to live with morphological homoplasy.
You may also agree that the distinction between Huperziaceae and Lycopodiaceae s.s., at least in spore ornamentation, is clearer than that among some families in eupolypods II we recognized. It reminds me of the excellent paper by Riibe et al. (2021) who transferred Diplazium praestans (Athyriaceae) to Hymenasplenium (Aspleniaceae).
No matter how morphologically similar between H. and L., their divergence is nearly as old as that between homo sapiens and the lobe-finned fish.
Best wishes, Libing
Li-Bing Zhang
Curator
Missouri Botanical Garden

This proposal was voted on during PPG Ballot 10 (voting period April 2024). A total of 81 votes were cast. There were 20 'Yes' votes (24.7%) and 61 'No' votes (75.3%). The proposal does not pass.


Release checksum verification instructions

Release checksum verification instructions
I went through the README files of the project but failed to find instructions on how to verify a released envoy binary using the checksum.txt.asc file. What I am missing is the public key but I am cannot find any reference as to where to download it from.
Maybe it would be nice to add a section in the README on how to verify a released envoy binary.
Thank you!

is this something you would be able to look into @timmotw ?

@phlax yes I could look into it but what is the URL of the key?

what is the URL of the key?

its not published anywhere atm - i just didnt get round to it
i did a bit of testing in terms of key management - not totally happy with current scheme but good enough i guess - im thinking it would probably be best to publish to a trust anchor - ie keybase or somesuch - and add to readme/release pages

I think the following key servers look like good candidates:
https://keys.openpgp.org/
https://keyserver.ubuntu.com/

sounds like a plan - let me dig out the key

yeah - that was the bit i wasnt happy with

Could Let's Encrypt be used at a later stage?

At least it verifies it fine; this is another issue on its own right I guess

Another solution would be publishing the key along the other release files. What do you think? It could be less complicated for some users and could render the need for instructions unecessary.

my ideal is that we keep a root key somewhere and use that to sign temporal keys that sign the actual release etc
i think that is kinda the scheme the distros have - but not entirely clear how that is setup

publishing the key along the other release files.

yeah i think we can just add it as a file to the repo now and link it with some instructions - would be nice to have a better scheme where it just says "trusted!"

if you can pr adding the asc as a repo file and perhaps update the README i would be happy to review - i think we can also use a fingerprint in any readmes etc
we can also add a link and/or explainer in the release messaging

@phlax here it is: https://github.com/envoyproxy/envoy/pull/30922

i just noticed ...
bcdc207948450f84e90fbd29d514285bce34141af49ddba5daa98fb0ca195dd9  /tmp/tmp5ts3tb33/bin/envoy-contrib-1.28.0-linux-x86_64
01a4a88bd369fcfa7637c131510394a653aa782571987fb2f7b2f4bbb493dd11  /tmp/tmp5ts3tb33/bin/debs.tar.gz
2639563ce9bc09b1ee6bd4731760b596c8ed1e474cdd83cc9df9364516e14367  /tmp/tmp5ts3tb33/bin/envoy-1.28.0-linux-x86_64
eb930e32ab5555643e09d11d490e392d0a790c5a80eb0b0ebacb1046bdbb114d  /tmp/tmp5ts3tb33/bin/envoy-1.28.0-linux-aarch_64
b6fb0c7c365f2348b32d086ea59b969e2b907e9c42a7f8819fd3adbb3fde0148  /tmp/tmp5ts3tb33/bin/envoy-contrib-1.28.0-linux-aarch_64

probs we should fix the tooling to get rid of the /tmp/... prefix - shouldnt matter to the checksums but not helpfulf if we change it and people build tooling around it

@phlax it is of upmost importance to remove the author(email) reference in the commit. I have just noticed it now.

not sure i follow

i just noticed ...
bcdc207948450f84e90fbd29d514285bce34141af49ddba5daa98fb0ca195dd9  /tmp/tmp5ts3tb33/bin/envoy-contrib-1.28.0-linux-x86_64
01a4a88bd369fcfa7637c131510394a653aa782571987fb2f7b2f4bbb493dd11  /tmp/tmp5ts3tb33/bin/debs.tar.gz
2639563ce9bc09b1ee6bd4731760b596c8ed1e474cdd83cc9df9364516e14367  /tmp/tmp5ts3tb33/bin/envoy-1.28.0-linux-x86_64
eb930e32ab5555643e09d11d490e392d0a790c5a80eb0b0ebacb1046bdbb114d  /tmp/tmp5ts3tb33/bin/envoy-1.28.0-linux-aarch_64
b6fb0c7c365f2348b32d086ea59b969e2b907e9c42a7f8819fd3adbb3fde0148  /tmp/tmp5ts3tb33/bin/envoy-contrib-1.28.0-linux-aarch_64

probs we should fix the tooling to get rid of the /tmp/... prefix - shouldnt matter to the checksums but not helpfulf if we change it and people build tooling around it

I only looked into 1.26.0 which does not include prefixes. Maybe the problem was introduced at a later version

not sure i follow

Can we remove the email of the commit author from the repo? There is a violation of policy which I would like to avoid. It is my mistake but I cannot do anything about it.

ah - hmm - i think this might be an artefact of me rescuing the build - ill check it on the tooling but probs ok

Can we remove the email of the commit author from the repo?

the main branch is immutable so nothing we can do about that im afraid

I really appreciate this and I am sorry to be you into that


bug in PLOT of stChromagram()

Hi,
you should add the following line after the Ratio calculation :
Ratio = chromaGramToPlot.shape[1] / (3*chromaGramToPlot.shape[0])
if Ratio < 1 : Ratio=1
I had Ratio equal to 0 that created problems with range afterwards.
best regards

Thank you, so much.
Change committed in master.


Linux porting

Is there some plan/activity to port this to linux ?
If not can you provide some guidelines for anyone willing to try ?

Hi Vincenzo!
Macaw was built around Apple's Cocoa which unfortunately doesn't exist for Linux. It would be great to have all the same API around another rendering engine, but this will be huge amount of work. If you want to try then you can start with examination of Macaw rendering system: https://github.com/exyte/Macaw/tree/master/Source/render. The idea would be replace all UIKit-dependent code with some Linux-based system (for example GTK).

Thanks Yuri, I will take a look.
I saw some project around claiming to build a swift CoreGraphics for Linux, I don't know if they are mature enough though.


Feed variables into other variables (limit scope of variables)

Your idea:
I'm opening this on behalf of a discord user so hopefully I have the request right-- could variables somehow have their scope limited to some degree, such that a variable could be reassigned throughout the brew?
The idea is to be able to do something like this:
[TraitMagicResistance]: The $[Name] has magic resist

[Name]: Sally

$[TraitMagicResistance]

[Name]: Barney

$[TraitMagicResistance]

and get output like this:
The Sally has magic resist

The Barney has magic resist

So the Name variable could be reused within another variable template.  To more precisely show the thinking of the requesting user, the TraitMagicResistance could take in another variable to fill slots, like this:
[TraitMagicResistance]: The $[Name] has magic resist

called like this
$[TraitMagicResistance][Sally]
$[TraitMagicResistance][Barney]

And I suppose with this latter syntax the trailing variables would just fill in from beginning to end if there were multiple.  Sort of like regex The $1 has magic resist substitution.

I anticipate that sometimes authors will want a variable insertion to occur dynamically, and other times will want the value captured as at time of declaration.

Wasn't this already discussed pretty heavily when designing the specs for variables? I feel like we decided to avoid this exact thing for a specific reason, but I can't recall at the moment.
In any case, something like this would involve basically a full rewrite of the Variables extension. I would put this at low priority for now.

Done for two main reasons: 1) to avoid recursion hell, and 2) to facilitate imperative interpreting.
Both are still good reasons.
The functionality suggested by $[TraitMagicResistance][Sally] shouldn't even involve further varnames. As Gazzok89 suggests, the functional form declaration should be more like [TraitMagicResistance]: The $1 has magic resistance. .. where the $1 represents to first fed-in parameter. This way each parameter can be re-used in the same template too, like $1 and $2 were sitting in a tree. $1 was hopeful, $2 not so much. $3 observed from a distance.. (The syntax suggested here is only a placeholder).
Whether these are called template variables, or madlib variables, or function variables .. eh, whatever works.
In any case, I agree .. we should let variables sit for a while before attempting to extend it. Let the bugs shake out. And remain open to a possible rewrite as a solution.

Yeah, I felt this was likely discussed heavily but I just couldn’t point to where, and I wasn’t really involved in variables discussion (by choice).

Talked about this again in gitter.  Giving some more examples of how little templates could be made:
[heroBlock]:
{{hero
## $[hero]
$[tagline]
}}

[hero]:Superman

[tagline]:Man of Steel

$[heroBlock]

[hero]: Batman

[tagline]: The Dark Knight

$[heroBlock]

or
[lootTable]:
##### Loot Table
| Item              | Roll  |
|:------------------|:-----:|
| $[loot1]          | 1     |
| $[loot2]          | 2     |
| $[loot3]          | 3     |
| $[loot4]          | 4     |
| $[loot5]          | 5     |
| $[loot6]          | 6     |

[loot1]: Pair of dice

[loot2]: Bottle of water

[loot3]: One torn sock

[loot4]: A magic card

[loot5]: A spoon

[loot6]: Twelve turtle doves

$[lootTable]


Use user.openshift.io to make the determination about openshift rather than route.openshift.io

Type of change

Bugfix

Description
For 0.31, we change the way openshift was detected.

the reliance on route.openshift.io also implied that the serviceaccount needed those permissions which meant the detection failed if the permissions were not present.
also propagared ENMASSE_OPENSHIFT from the enmasse-operator to the console-server.

Checklist

[ ] Update/write design documentation in ./documentation/design
[ ] Write tests and make sure they pass
[ ] Update documentation
[ ] Check RBAC rights for Kubernetes / OpenShift roles
[ ] Try your changes from Pod inside your Kubernetes and OpenShift cluster, not just locally
[ ] Reference relevant issue(s) and close them after merging
[ ] Update CHANGELOG.md

@enmasse-ci run tests testcase=*ConsoleTest test_only

:heavy_check_mark: Test Summary :heavy_check_mark:
TEST_PROFILE: pr
TEST_CASE: *ConsoleTest
TOTAL: 60
PASS: 55
FAIL: 0
SKIP: 5
BUILD_NUMBER: 59


Issue with shared library cmake example

Hey there,
I was going through the examples to learn about cmake & i saw that for shared library examples if we are building the executable it will show following
D:\Study\compiler\software\src\Hello.cpp(3): fatal error C1083: Cannot open include file: 'shared/Hello.h': No such fil
e or directory [D:\Study\compiler\software\build\hello_library.vcxproj]

Unfortunately I haven't tested on windows as I don't have access to a visual studio. Were you able to get this working in the end?


Task for imprort from CSV isn't saved correctly.

System information:

Operating system (distribution) and version win 10 x64
DBeaver version 6.2.2

Describe the problem you're observing:
Task for "Import from CSV" isn't saved correctly. Import info in Task window is empty
Steps to reproduce, if exist:

Start import from CSV to some table from Context menu in DB nsvigator
Choose CSV, Save task
Open task from DB tasks tab -> Import info is empty - no table here, import couldn't be started

Fixed

Verified, but ticket 6883 is still reproducing.


Fish highlights only apply if the Nix attribute sets are nested a certain way

For example, to get Fish highlighting, this works:
programs.fish = {
  functions = {
    my-function = ''
      echo "some shell script"
    '';
  };
};

# this structure also works
programs.fish.functions = {
  my-function = ''
    echo "some shell script"
  '';
};

However this does not work:
programs.fish = {
  functions.my-function = ''
    echo "some shell script"
  '';
};

This is due to a mistake in how the hmts-path directive was implemented. Shouldn't be too hard to solve, expect a fix this weekend.

This should be fixed in #14, can you confirm?

It works! Thanks for the quick turnaround on these. This is a really awesome plugin!


zstd_legacy.h is missing - include attempted when ZSTD_LEGACY_SUPPORT is defined

ZSTD_LEGACY_SUPPORT if not defined is defined to 1 in zstd_decompress.c
Then later in the file:
#if defined(ZSTD_LEGACY_SUPPORT) && (ZSTD_LEGACY_SUPPORT==1)
include "zstd_legacy.h"
#endif
However, zstd_legacy.h isn't present on github.

It is present in the legacy folder
The provided Makefile is supposed to automatically include this directory.

Ah, my mistake - I was updating ZStd in lzbench, which doesn't have the folder.
How about defaulting ZSTD_LEGACY_SUPPORT to 0, so that the legacy folder is only required if specifically desired?

It's a matter of preference.
I'm not sure whichever option is better "by default".
So I opted for the "safer" one, so that people / projects having used Zstd prior version don't get accidentally blocked.

Sure, understood. I was assuming from the major version being zero, that api changes were assumed to be okay at any time. That is minor issue with semantic versioning before the first major release.
Thank you.

Actually, thinking again about it, something I could do and feels logical :

when built using the default provided Makefile : set ZSTD_LEGACY_SUPPORT to 1
when file is just being copy pasted into another project and no  ZSTD_LEGACY_SUPPORT is provided, set it to 0

This way, when including zstd into another project, the expectation, in general, is to not need additional legacy files.

That looks to be a very good solution to me. It is difficult to balance having the code out there for people to use and report bugs etc, before committing to a major version. Keeping the main lib folder as current and self sufficient would make adoption easy, while minimizing the penalty for early adoptors to migrate to changes.

pushed into dev branch


Add ability to set Timezone in config

Using the .env file, we can change the timezone used for Twig.
I use Laravel Valet, and setting the timezone via php.ini does not work. But this does. 😀

I think I can replace both instances of $twig->getEnvironment()->getExtension(CoreExtension::class) with a variable, but I'm not 100% sure. I've never used Twig so I don't know if that works or not.
I also couldn't find any tests relating to the display of the date & time, so I was not sure where exactly to insert a test for this.

Fixed errors and merged manually.
Thank you for the PR!

Thanks @PHLAK! Do you have a plan for when you'll do the next release?

I was just debating if I wanted to push this out as it's own update today @mikemand. I think I might.

No rush. I will just be happy to finally not have to do math every time I have to check when a file was last updated, LOL.

Here you go: https://github.com/DirectoryLister/DirectoryLister/releases/tag/3.0.5

You are the best! Thanks!


Credentials in URL

We have a Jenkins server that's behind a global HTTP basic authentication scheme for the whole domain and has a web form for logging into Jenkins.  When I try to add the HTTP basic authentication I get and error saying "[Fail] Credentials should not be embedded in the url.  Use the above form instead."   Is there any possibility to having both a HTTP basic authentication and form based authentication?
Thanks!

hi, Are there some solutions to fix this?

Currenttly not
But I can look after my current issue

I would love to use this plugin, but I have the same issue, not possible with basic authentication as far as I see.

I will have a look
do you have test environment to test against?

No sorry we have no public test environment, sorry, only our internal company instance.
I can reproduce the issue without valid jenkins url/credentials like this

Ok
i have to setup a test environment for this then.
so for clarifying. If you visit the jenkins via browser it will ask with basic auth to login and you see the jenkins login screen or will you be logged in then?

The web server requires basic auth and jenkins requires user login which are not related to each other as far as I see.
Visiting via browser prompts for basic auth and loads the login page

Ok
i will setup this on my machine and test it. But it take some time. I guess I can test it in 1-2 week.
But I will look into it for one of next releases

@lee-peuker Sorry for bothering but do you know if the basic auth is handled via a proxy?
maybe you can check with curl -v and check the header in response.
If it is a proxy it could maybe help to configure proxy in IDE with credentials.
But I will setup an environment next week too with a simple basic auth front of Jenkins

@MCMicS Hey no problem. We have a reverse proxy (nginx) in front of of Jenkins, which uses the standard auth_basic module and proxy_pass to connect to Jenkins.

If it is a proxy it could maybe help to configure proxy in IDE with credentials

I will try this out.

Can you share part of your proxy settings?
Do you modify headers or simply use proxy_pass or any header modification?
What plugins are installed on jenkins. maybe something like https://plugins.jenkins.io/reverse-proxy-auth-plugin/

I have removed and censored a few things from the nginx config
server {
    listen 443 ssl http2;

    server_name [...];

    root [...];

    access_log /var/log/nginx/ci.access_log;
    error_log  /var/log/nginx/ci.error_log;

    proxy_intercept_errors  on;

    ignore_invalid_headers off;
    sendfile off;

    location / {
        auth_basic "Protected Area";
        auth_basic_user_file [...];

        allow [...];
        satisfy any;

        if (!-f $request_filename) {
            proxy_pass          http://app_server;
            break;
        }
            proxy_pass          http://app_server;
            break;

        proxy_redirect http:// https://;
        proxy_http_version 1.1;
        proxy_max_temp_file_size 0;

        client_max_body_size       10m;
        client_body_buffer_size    128k;

        proxy_connect_timeout      90;
        proxy_send_timeout         90;
        proxy_read_timeout         90;
        proxy_request_buffering    off; # Required for HTTP CLI commands in Jenkins > 2.54

        proxy_set_header Host $host:$server_port;

        proxy_set_header        X-Real-IP       $remote_addr;
        proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header        Authorization   "";
        proxy_redirect          off;

        # workaround for https://issues.jenkins-ci.org/browse/JENKINS-45651
        add_header 'X-SSH-Endpoint' 'jenkins.domain.tld:50022' always;

        #mixed content jenkins to non https
        proxy_set_header X-Forwarded-Proto $scheme;
    }

    location /themes {
        root /var/lib/jenkins/themes;
    }
}

I think relevant is probably proxy_set_header        Authorization   ""; -> jenkins knows nothing about the basic auth done by nginx
If I search the installed plugins I found no match containing the word proxy or reverse.

thanks for this
i try to setup this. and yes it looks like removing existing Authorization header. so jenkins wiull not know the auth which is configured in IDE
I try to find a a solution. Is a change on then proxy configuration püossible. So if I send a different header which will be delegated to the jenkins after the basic proxy auth.,

Is a change on then proxy configuration püossible

Hm, maybe, but I am not sure. I can ask when I have something concrete to propose.

Hello,
I hope this may help you identify the root cause
I'm facing the same issue and  in my opinion it's not related to ingress, I'll explain.
(sorry I didn't yet find the exact versions we were using)
In June 2023 I created an API token for my account and I still, now, can use it with basic auth.
This API token is no more visible in jenkins UI.
In July 2023 we did an upgrade of Jenkins (I was not able to figure out which version we had and to which we upgraded)
Starting from that date, all newly created API tokens are not working with basic auth.
In September 2023, we did another upgrade (current Jenkins version 2.414.1), and the issue remains.

@MerlinFTW
Your Api token not works anymore?
if it is please file a new issue. This issue is about credentials in url for infront proxy for example
But API toekn will work for current jenkins and plugin Versions

The API token I created before June 2023 is still working (basic auth) but it does not appear in Jenkins UI.
The new API tokens I create (since upgrade to 2.401.2) are not working.
Do you think it require a new issue ? if so I'll open one

Yes please. It seems than not belong to credentials in url


Members of structure

Some more detail, such as brief description link to page giving fuller description & usage required. Also details about how and when and why the spooler calls the functions and what it expects the UI to do.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: b287bef8-87ec-19e4-b7aa-7aa5b9896198
Version Independent ID: 0ceecce9-a675-c929-1025-2c20e9a562fd
Content: _MONITORUI (winsplp.h) - Windows drivers
Content Source: wdk-ddi-src/content/winsplp/ns-winsplp-_monitorui.md
Product: windows-hardware
Technology: windows-devices
GitHub Login: @barrygolden
Microsoft Alias: bagold

@JonAxtell, thanks for your feedback and for pointing out the missing information.
I'm opening an internal work item to fill in missing member descriptions and update the MONITORUI structure topic.

@JonAxtell, thanks for your feedback and for pointing out the missing information.
I'm opening an internal work item to fill in missing member descriptions and update the MONITORUI structure topic.

Updated topic: filled in missing member function pointer descriptions.


Intellisense shows type any for inferred field in object.

TypeScript Version:  3.1.1

Search Terms:
intellisense any recursion
Code
type Check<T> = (value: any) => value is T;
type Static<T> = T extends Check<infer U> ? U : never;
type Interface<T extends { [index: string]: Check<any> }> = Check<{ [P in keyof T]: Static<T[P]> }>;

type IPlayer = Static<
	Interface<{
		name: Check<string>;
		data: Interface<{
			score: Check<number>;
		}>;
	}>
>;

Expected behavior:
Hovering over IPlayer should show:

Actual behavior:
Hovering over IPlayer shows the following:

Playground Link:
Playground Link
Related Issues:
N/A

If I index the type further, it seems to resolve correctly.
type IPlayerData = IPlayer["data"];

Hover:

This issue also effects the intellisense when trying to index the object of that type.
Also, I'm not really sure what to title this issue. If anyone has a better phrase for what's going on here, I'm happy to edit the title.

When you compile, does the emitted .d.ts file also show any?

Interface creates what we call within the compiler a "reverse mapped type". These types lazily compute their children. This is done to avoid blowing up when a type anonymously reverse maps itself (as there's nothing to name and reuse to terminate the circularity with). In declaration emit/node printing, we currently limit printing of these types to 1 level deep.
Duplicate of #23897 BTW - I'd chime in there.


Call back for custom regularizer Coefficient

I, for my application have created a custom Regularizer. And I wish to vary the "weight decay constant" after every certain number of epcohs. Is this implementation possible with callback ?
Example code:
from keras import backend as K
def l1_reg(weight_matrix):
return 0.01 K.sum(K.abs(weight_matrix))
model.add(Dense(64, input_dim=64,
kernel_regularizer=l1_reg))
model.compile()
Now I want to vary the 0.01 in the l2_reg, after every certain number epcohs.
I have tried a similar approach which was followed for decaying learning rate. But I am not able to implement using that idea, correctly. Any suggestions would be really helpful.
Thanks!

As long as your updates are low-frequency (e.g. once per epoch) you could do:
from keras import backend as K
import keras

l1_rate = K.variable(0.01)

def dynamic_l1(w):
   return l1_rate * K.sum(K.abs(w))

cbk = keras.callbacks.LambdaCallback(on_epoch_end=lambda epoch, logs: K.set_value(l1_rate, 0.01 * (epoch + 1)))

model.fit(..., callbacks=[cbk])

This would set the rate to 0.01 * (epoch + 1) after every epoch (increase it by 0.01 after each epoch)
Warning: untested etc.

@fchollet Thank You! I have checked its working !! I have tried something on this line.
##Function to implement the decay  rate annealing
def decay_schedule(epoch):
#Initial Learning rate
d_r = 0.01
if epoch > 100:
d_r = 0.00001 * d_r
elif epoch > 70:
d_r = 0.0001 * d_r
elif epoch > 50:
d_r = 0.001 * d_r
elif epoch > 30:
d_r = 0.01 * d_r
    print ('Decay rate:', d_r)
    return d_r

class DecayRate_Controller(keras.callbacks.Callback):
def init(self, controller):
super().init()
self.controller = controller
    def on_epoch_begin(self, epoch, logs=None):
            d_r = self.controller(epoch)
            K.set_value(d_rate, d_r)


Use Github raw domain as content source

The domain raw.githubusercontent.com does not impose rate limits. In fact using the regular github.com domain to access a static asset (like a PNG) using cURL redirects to this raw domain:
curl -Is "https://github.com/{ORG}/{REPO}/raw/{BRANCH}/docs/image.png"

HTTP/1.1 302 Found
Server: GitHub.com
Date: Thu, 22 Jun 2017 18:31:55 GMT
Content-Type: text/html; charset=utf-8
Status: 302 Found
Cache-Control: no-cache
Vary: X-PJAX, Accept-Encoding
X-RateLimit-Limit: 100
X-RateLimit-Remaining: 100
Access-Control-Allow-Origin: https://render.githubusercontent.com
Location: https://raw.githubusercontent.com/ORG/REPO/BRANCH/docs/image.png
X-UA-Compatible: IE=Edge,chrome=1
X-Runtime: 0.017536
Strict-Transport-Security: max-age=31536000; includeSubdomains; preload
X-Content-Type-Options: nosniff
X-Frame-Options: deny
X-XSS-Protection: 1; mode=block
X-Runtime-rack: 0.020754
X-Served-By: 7d2a2d05162492046d9960cdbc326796
Age: 0
X-GitHub-Request-Id: D5DB:4C9C:2CFB8EA:4161CAD:594C0D1B

When we request the raw domain content:
curl -Is "https://raw.githubusercontent.com/ORG/REPO/BRANCH/docs/image.png"

HTTP/1.1 200 OK
Content-Security-Policy: default-src 'none'; style-src 'unsafe-inline'
Strict-Transport-Security: max-age=31536000
X-Content-Type-Options: nosniff
X-Frame-Options: deny
X-XSS-Protection: 1; mode=block
ETag: "fd51ed6a84d1ee36c3b0af994dea8c32c785bb29"
Content-Type: image/png
Cache-Control: max-age=300
X-Geo-Block-List:
X-GitHub-Request-Id: AB1E:1776:F1BF31:F95F97:594C0D02
Content-Length: 18411
Accept-Ranges: bytes
Date: Thu, 22 Jun 2017 18:34:09 GMT
Via: 1.1 varnish
Connection: keep-alive
X-Served-By: cache-iad2125-IAD
X-Cache: HIT
X-Cache-Hits: 1
X-Timer: S1498156449.322911,VS0,VE1
Vary: Authorization,Accept-Encoding
Access-Control-Allow-Origin: *
X-Fastly-Request-ID: a440cac380cd0e67b5ebdc40c5d8a4185be58bf1
Expires: Thu, 22 Jun 2017 18:39:09 GMT
Source-Age: 159

Per these HTTP responses we can see there is no rate limit imposed. As we are only targeting public repos it makes sense to switch all our URLs to this domain.
This starts work related to #6 and is a non-breaking change we can implement today.


Unhandled Rejection (Error): invalid BigNumber string (argument="value", value=".0001", code=INVALID_ARGUMENT, version=bignumber/5.0.8)

Seeing this error when i try to enter a value(0.0001) along with the set purpose string and submit. Any idea what i might be missing here

How are you calling the set purpose function? Can you show the code..


Update MiMa configuration

Right now only one module (circe-jackson25) is set up to have binary compatibility check, since it's the only module that existed before today's 0.6.2 release. Now that releases are out in the world for all four modules, it'd be great to have MiMa checking for all of them.

Fixed in #6.


Voice Gate Ping

This PR's purpose is to ping new users in the voice_verification channel so that they can figure out why they aren't able to talk.  Currently, we have a lot of people confused about the new system. There are multiple people popping in and out of the voice channel thinking that there's something wrong with their connection or thinking they got specifically suppressed.
By pinging the user in the voice_verification channel (along with a little explanation on how to use it), this should reduce the number of folks asking.
The reason that this is only sending a ping in the channel and not a direct DM is that a fair number of people may have their DMs turned off.  Another reason is that if they are joining a voice channel, they'll already be looking in that section of the channels, and a ping that's right in front of their eyes should get them looking in the right place.
Closes #1261

We've agreed with @MrHemlock that I'd help address some of the feedback so that we can merge it soon.

I'm spying a race issue here, when users might run into the pathological case when discord repeatedly spam joins/leaves the voice channel. This can result in over 6 joins/leave events per second, and while I'm not sure if the issue is fixed and do not have a way to reproduce it under my control, I think we should be careful and address that case.
At the moment, the user-ping lockout in the cache is split over a send message and a wait-ready event, which makes it especially prone to getting re-entered in that situation, especially since there have been incidents where the bot takes 1-2 seconds to respond to things, depending on load and events happening elsewhere. We probably need a better lockout.
Would it be reasonable to have a sentry "sending message" value to set into redis while we wait to send the message?

@bast0006 You're definitely right that there's still plenty of opportunity for racing, both with concurrent voice events and the on_message too. I'd consider it pathological because I wasn't able to create such a situation myself, but one needs to consider that in the production environment the event loop is busier, which I imagine creates larger "windows" of opportunity for things to go wrong.

Would it be reasonable to have a sentry "sending message" value to set into redis while we wait to send the message?

Using Redis already comes with the problem that you can only interface with it asynchronously, so I don't think it's a bulletproof solution. A lock probably makes more sense, because you never have to await anything to acquire it.
I think I have an idea for how to solve it using a smart locking util that @MarkKoz wrote for another PR. It should allow us to prevent both the message & the voice state listener from runninng concurrently for the same user, using a decorator, which avoids having to add extra logic to the actual code. I'm planning to look at it soon.

Everything is working as intended.  Pushing her through.


Run Anserini baselines on AI2 citation prediction task (OpenResearch corpus)

The corpus contains 45M academic papers and the task is to predict which papers should be cited in a paper given its title and/or abstract as input.
https://api.semanticscholar.org/corpus/
We should compare our results against the BM25 implementation from this paper (table 2):
https://arxiv.org/pdf/1802.08301.pdf

Update: I was able to reproduce the F1@20 from https://arxiv.org/pdf/1802.08301.pdf but not their MRR.
F1@20: 0.059 (Anserini BM25) vs 0.058 (Whoosh BM25, Bhagavatula et. al, 2018)
MRR: 0.148 (Anserini BM25) vs 0.218 (Whoosh BM25, Bhagavatula et. al, 2018)
I will send a PR as soon as I find the reason for the MRR discrepancy.

If it isn't too hard, try Whoosh also just to isolate the differences?

Good idea, I will try it.

One potential cause for the difference in MRR is that we only use the title as query whereas Bhagavatula et. al (2018) mention in appendix D that they use title AND abstract as query.
They use whoosh's key_terms_from_text method to extract terms from the query in order to make it shorter. Does anyone know an equivalent function in Anserini?

Nope, although we can use Whoosh to generate the queries, and then throw into Anserini? If that turns out to be the difference, we can reimplement in Java?

Sound good!

I found the difference: we should not return the same that doc that originated the query. Giving title only as a query to Anserini (as opposed to title+abstract from Bhagavatula et. al), the numbers are:
F1@20: 0.063 (Anserini BM25) vs 0.058 (Whoosh BM25, Bhagavatula et. al, 2018)
MRR: 0.246 (Anserini BM25) vs 0.218 (Whoosh BM25, Bhagavatula et. al, 2018)

I created PR to reproduce these results:
https://github.com/castorini/anserini/pull/682

Closing this as the BM25 baseline was successfully implemented. Thanks @bazingagin !


Not able to list sidecar containers by their name

acorn version v0.5.1-82-g652e22e7+652e22e7
Steps to reproduce the problem:
Deploy apps with containers that have sidecar.
Acorn file used:
args: {
fileName: "hello.html"
fileContent: "content"
}
containers: {
    frontend: {
        build: "."
        cmd: ["/root/start.sh", args.fileContent,args.fileName]
        dirs: {
            "/scratch": "volume://scratch-data"
        }
        sidecars: {
            sidecar: {
            ports: publish: "80/http"
            image: "nginx"
            dirs: {
                "/usr/share/nginx/html/test1": "volume://scratch-data"
            }
          }
        }
}
}
volumes:
{
"scratch-data": {}
}

Once the app us deployed ,we are not able to  list sidecar containers by their name.
 % acorn containers test1
NAME                                      APP       IMAGE                                                                     STATE     RESTARTCOUNT   CREATED    MESSAGE
test1.frontend-8584688cbc-9wdt8           test1     sha256:ee451425b88b30205998e298b96470bacb7a9eaa75d9124f759d9cf314af8ba8   running   0              3m9s ago
test1.frontend-8584688cbc-9wdt8.sidecar   test1     nginx                                                                     running   0              3m9s ago

 % acorn containers test1.frontend-8584688cbc-9wdt8.sidecar
  ✗  ERROR:  appinstances.internal.acorn.io "frontend-8584688cbc-9wdt8" not found

This might be addressed by the PR to fix execing into sidecars

sidecar exec PR I mentoined: https://github.com/acorn-io/acorn/pull/1317

Tested with acorn version -v0.6.0-142-ga0b25aa9+a0b25aa9
Able to list sidecar containers by name.
acorn containers myside
NAME                                    APP       IMAGE     STATE     RESTARTCOUNT   CREATED   MESSAGE
myside.nginx1-d548d6844-cj29q           myside    nginx     running   0              24s ago
myside.nginx1-d548d6844-cj29q.sidecar   myside    ubuntu    running   0              24s ago

 % acorn containers myside.nginx1-d548d6844-cj29q.sidecar
NAME                                    APP       IMAGE     STATE     RESTARTCOUNT   CREATED   MESSAGE
myside.nginx1-d548d6844-cj29q.sidecar   myside    ubuntu    running   0              37s ago
sangeethahariharan@Sangeethas-MBP sidecar %


Add Crewed supersonic science to cockpits

Still untested! I'll test them properly later. As it's my first venture into Kerbalism configs, decided to put it here already in case something obvious is wrong.
This depends on the removal of all other science from the Earth Flying low in RP-0, soon to be commited

I have tested it ingame, and the experiments work. I'd like some input on the values and parameters for balancing though.

Looking at this PR through the GH web interface, there appears to quite a few places with mixed spaces and tabs.


Update azure-site-recovery-overview.md

removed preview notes

@ronmiab
IMPORTANT: When the changes are ready for publication, add a #sign-off comment to signal that the PR is ready for the review team to merge.
#label:"aq-pr-triaged"
@MicrosoftDocs/public-repo-pr-review-team


Nix: No such file or directory: '~/.cache/pre-commit/...python3.10/bin/ruff'

I'm working on a project which uses ruff via pre-commit.  I've simplified the .pre-commit-config.yaml to the following:
---
repos:
  - repo: meta
    hooks:
      - id: check-hooks-apply
      - id: check-useless-excludes

  - repo: https://github.com/charliermarsh/ruff-pre-commit
    rev: 'v0.0.229'
    hooks:
      - id: ruff

When I run pre-commit run --all-files I get the following:
 ❯ pre-commit run --all-files
[INFO] Initializing environment for https://github.com/charliermarsh/ruff-pre-commit.
[INFO] Installing environment for https://github.com/charliermarsh/ruff-pre-commit.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
Check hooks apply to the repository......................................Passed
Check for useless excludes...............................................Passed
ruff.....................................................................Failed
- hook id: ruff
- exit code: 1

[Errno 2] No such file or directory: '/home/matt/.cache/pre-commit/repo2mikycmg/py_env-python3.10/bin/ruff'

Which is weird, because that file definitely exists.
This is on my Linux machine (NixOS 23.05).  I've been working around it by just doing the linting on macOS, but that's kind of a pain.  Any idea what I can do to fix this?

Could this be related to https://github.com/charliermarsh/ruff/issues/1699?

Seems likely, I have that same experience:
❯ nix-shell -p python311
[nix-shell:~/2023/01/21]$ python -m venv venv
[nix-shell:~/2023/01/21]$ source venv/bin/activate

[nix-shell:~/2023/01/21]$ pip install ruff
Collecting ruff
  Using cached ruff-0.0.229-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (4.3 MB)
Installing collected packages: ruff
Successfully installed ruff-0.0.229

[nix-shell:~/2023/01/21]$ ruff
bash: /home/matt/2023/01/21/venv/bin/ruff: No such file or directory

There is a workaround, but it's really really ugly:
rm '<offending ruff binary>'
ln -s $(which ruff) '<offending ruff binary>'

That workaround works, but obviously only until pre-commit regenerates that cache! It would be nice a more proper fix for this at some point :)

The beginnings of a fix might look like invoking autoPatchelf in the shell hook.
An example .pre-commit-config.yaml that uses ruff
---
repos:
  - repo: https://github.com/charliermarsh/ruff-pre-commit
    rev: 'v0.0.229'
    hooks:
      - id: ruff

An example flake.nix that add pre-commit and a libgcc which ruff binary depends on. Note I don't regularly use flakes, this works but may be bad copy+pasta. The important bits are in the shellhook and in nativeBuildInputs
{
  description = "Dev env";
  inputs = {
    nixpkgs.url = "github:nixos/nixpkgs/nixpkgs-unstable";
    flake-utils.url = "github:numtide/flake-utils";
    flake-compat.url = "github:edolstra/flake-compat";
    flake-compat.flake = false;
  };
  outputs = {
    nixpkgs,
    flake-utils,
    ...
  }:
    flake-utils.lib.eachDefaultSystem (system: let
      pkgs = import nixpkgs {
        inherit system;
        overlays = [];
      };
    in {
      devShells.default = pkgs.mkShell {
        name = "dev-env";
        buildInputs = with pkgs; [
          pre-commit
          glibc.libgcc
        ];
        nativeBuildInputs = [ pkgs.autoPatchelfHook ];
        shellHook = ''
          pre-commit install
          # oddly the python pre-commit hook doesn't actually install things during install
          # here's a hack to force ruff into the pre-commit cache
          pre-commit run
          # pre-commits output is a bit verbose when it succeeds, choose your own output wrangling
          patch=$(autoPatchelf ~/.cache/pre-commit/)
          if [[ $? -ne 0 ]]; then
            echo "$patch"
            exit 1
          else
            echo "patched ~/.cache/pre-commit"
          fi
        '';
      };
    });
}

Inspiration from https://sandervanderburg.blogspot.com/2018/10/auto-patching-prebuilt-binary-software.html

I believe this issue can be closed and, perhaps, tracked somewhere more nix specific.

I agree.  This repo is a way to access the problem, but it doesn't itself have a problem.  Let's close it.

If I'm understanding correctly, the "right way" to fix this requires:

somebody who cares about nix...
...to go fix the way pip (a python package manager) installs ruff (a program written in rust)

But a big part of the reason to be a 1-type person is because you don't want to deal with 2-type problems anymore.  Maybe our hero is out there.  Maybe they'll fix it the right way.  But I'm not going to hold my breath, I'm going to focus on finding a nicer workaround because I'd rather rely on fewer types of binary cache anyhow.
My initial explorations of https://github.com/cachix/pre-commit-hooks.nix indicate that it wants to replace pre-commit.  It will say things like this:

pre-commit-hooks.nix: WARNING: Refusing to install because of pre-existing .pre-commit-config.yaml

Translate .pre-commit-config.yaml contents to the new syntax in your Nix file
see https://github.com/cachix/pre-commit-hooks.nix#getting-started
remove .pre-commit-config.yaml
add .pre-commit-config.yaml to .gitignore

My team would not like it if I made that change (they're not all nix users).  But perhaps...
A. ruff can be added to pre-commit-hooks.nix
B. pre-commit-hooks.nix can be... configured? forked? such that it can comfortably share a repo with pre-commit, and instead just overrides certain hooks with its own--which I assume do not involved asking pip to build ruff.
This would solve my problem, but maybe it's overkill?  Maybe it won't solve your problem?  I'm curious if others think it's a sane approach.

For anyone coming here still looking for a solution, here's what worked for me:

Delete the cached ruff (eg: if you ran the hook before installing it via nixpkgs)
Rerun pre-commit (it will not used the cached one, and instead use your installed and correctly linked binary).


Rewrite the clearart method in order to send write events to the plugins.

Currently, beet clearart doesn't notify plugins that some files are modified. This can be a problem when for example using the beets-check plugin because checksums aren't recomputed and the modified files are then detected as not intact during subsequent operations.
I finally settled on fixing this by using the try_write method which allows to simplify the code and centralizes the event sending so that this functionality isn't left out in the future. However, by doing that, beet clearart doesn't delete the image tag, but just erases its value. I don't know if this could be an issue.
The beets-check plugin is the only one I know which needs to be aware of a beet clearart operation so I don't know if I can write tests for this. Can external plugins be called in the test suite?

Cool! Thank you for simultaneously simplifying the code while making it more correct. :sparkles:
A test for this could just assert that the appropriate event gets sent (using a dummy plugin) when the art is cleared. Would that work?
It would still be nice to remove the art tag altogether. Maybe we could refine the interface for Item.write and Item.try_write so that any Nones in the tags dict mean "remove this tag from the file"? Or some other way of specifying some tags to remove during the write.

I will try to come up with a test soon. I also looked at how tags could be removed from the file with the write method, and it seems that this is actually already the case, with the exact same convention None means delete*, so everything is already all good in that regard.
*See: https://github.com/sampsyo/beets/blob/master/beets/mediafile.py#L1502

Nice! That should do it. Thanks for sorting that out.

I added 2 commits with a first shot at testing event sending.
However, the test suite fails, here's the output for a single test:
FAIL: test_delete_art (test.test_mediafile.WavpackTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/nathdwek/opt/beets/test/test_mediafile.py", line 92, in test_delete_art
    self.assertEqual(eventConsumer.popEvent("write"), True)
AssertionError: False != True
-------------------- >> begin captured logging << --------------------
beets: DEBUG: Sending event: write
beets: DEBUG: Traceback (most recent call last):
  File "/home/nathdwek/opt/beets/beets/mediafile.py", line 1349, in __init__
    self.mgfile = mutagen.File(path)
  File "build/bdist.linux-x86_64/egg/mutagen/_file.py", line 222, in File
    fileobj = open(filename, "rb")
TypeError: coercing to Unicode: need string or buffer, NoneType found

beets: ERROR: uncaught Mutagen exception in open: coercing to Unicode: need string or buffer, NoneType found
beets: ERROR: error reading None: None: coercing to Unicode: need string or buffer, NoneType found
--------------------- >> end captured logging << ---------------------

I'm willing to investigate the assertion fail, but I've got no idea on how to take the mutagen errors. (Regarding the assertion error, the previous fix works on my machine (TM) with regular, non dummy files).
I'll probably still try to tackle this anyway tomorrow or in the rest of the week, but any help with mutagen and fixtures would be appreciated. :pray:

That's certainly odd. I don't quite see what's going wrong here, but I do think you might be testing at the wrong level. You want to test the beets.art.clear() function, which is actually at a different level of abstraction from MediaFile. Testing there won't trigger your new code.
Perhaps you want to put a test in test_art.py or test_embedart.py, in the approximate style of ArtSimilarityTest?
The event-capturing plugin fixture looks good, though!

Oh you're totally right. I'll give it another try.

Just a quick update: I've got some family stuff getting in the way of writing the tests and those are proving to be a little more challenging then expected, but I should eventually be able to sort this out.

Awesome! Thanks for sticking with it.

I'm sorry but I really can't find a way to write a good test for this. One of the main problem is that I struggle to correctly load a dummy plugin if it doesn't reside in beetsplug/, and I don't really like putting test code in there. Even then, my best attempt that involves a test plugin residing in this folder still feels really chunky and inelegant. Do you have a real interest/need to thoroughly test the plugin observer pattern? (Is it worth refactoring the code and putting in place the infrastructure to do this?) If so, could you share some leads because I'm seriously hitting a wall now.

No worries! Let's merge without a test for now and, later, work out the details. If you're interested, we can cook up a clean strategy in a new issue.
In the mean time, would you mind rolling back the current test code? With that and a changelog entry, we can merge now.

All right let me finalize this then. Do you prefer history rewrite or revert commits?

I went with rewriting history and used this as an opportunity to cleanly rebase on top of master (also because I wanted to try rebasing rather than merging as a way to contribute.). I hope it's ok for you.
Also I took care of the changelog. I didn't put a reference to the embedart plugin at the beginning because I think the fix happened at the core level, I don't know if it's how you'd assess it.
It makes me a bit uncomfortable plugging in my name like that in the changelog :blush:. but at least now hopefully you'll juste have to presse the big red merge button. Again, I hope it's ok this way for you.
As always thank you for your responsiveness. I kept the raw commits of my experimentation with tests on another branch, I will probably try to have a fresh look at this some day.

This looks perfect! Rewriting the history is just fine (for the record, either way is cool with me). And you're absolutely right to thank yourself; you deserve at least that. :smiley:


Free Provider Gemini-* models don't work

@tarasglek I get 500s (nothing in the response I can debug) when I try to use the Gemini models via the Free Provider:

it's working for me atm. Our error reporting could be better here. Next time this happens open up network tools to see how the error was reported. Openrouter usually has something helpful for debugging, just too many different ways of reporting the error


Deprecate and remove bufferWhen and windowWhen

It looks like bufferWhen(() => a) is the same as buffer(defer(() => a)) and windowWhen(() => a) is the same as window(defer(() => a)).
bufferWhen and windowWhen exist because it was part of getting parody for RxJS v4 (at the time v2) functionality (while splitting buffer into different operators). It seems like these two operators are just bloat and unnecessary.
Alternatively, we could have buffer and window optionally accept functions as arguments, however, I'm generally against this.

If using defer achieves exact same behavior, I'm all for removing operator.

My understanding is that in bufferWhen(() => a) is not the same as buffer(defer(() => a)), and that their behaviours are very different, as in bufferWhen, every time the closingSelector observable emits a value or completes, the closingSelector observable will be unsubscribed from, recreated and resubscribed to, whereas in buffer, the closingNotifier observable is only every subscribed to once
For example, bufferWhen(() => a) behaves vastly different to buffer(defer(() => a)) in this example:
import { interval, timer, defer } from "rxjs";
import { buffer, bufferWhen } from "rxjs/operators";

const a = timer(1000);
const source = interval(200);

source // buffer(defer(() => a))
  .pipe(buffer(defer(() => a)))
  .subscribe(
    value => console.log("using buffer with defer:", value),
    error => console.error("using buffer with defer:", error),
    complete => console.log("using buffer with defer:", "complete")
  );

// using buffer with defer: [0, 1, 2, 3]
// using buffer with defer: complete

source // bufferWhen(() => a)
  .pipe(bufferWhen(() => a))
  .subscribe(
    value => console.log("using buffer when:", value),
    error => console.error("using buffer when:", error),
    complete => console.log("using buffer when:", "complete")
  );

// using buffer when: [0, 1, 2, 3]
// using buffer when: [4, 5, 6, 7, 8, 9]
// using buffer when: [10, 11, 12, 13, 14]
// using buffer when: [15, 16, 17, 18, 19]
// ...

Ben says he's wrong, Matt concurs.


add Cantonese localisation

also fixed https://github.com/backmeupplz/shieldy/commit/e68fbf886d27ece9ac7a24f168be2bb3e90d3259 breaking https://github.com/backmeupplz/shieldy/commit/4a94f88677559c9176f1576414ca780e80002aba

Thank you a lot!


Error when using latest version of pip

See https://github.com/poise/poise-python/blob/master/lib/poise_python/resources/python_package.rb#L41
I ran into error, since pip has new releases recently
STDERR: The python_package resource requires pip >= 6.1.0, currently 10.0.0b2

it should check something like ^[0-6]\\.0 so it does not match 10

This is breaking our build as well. Any way to work around this?

I added python_runtime block with a pip version in the our recipe, as a temp fix. It works for me: python_runtime '2.7' do   pip_version '9.0.3' end

I encountered this error and worked around it by adding the indicated pip_version line to my recipe.
case node['platform_family']
when 'amazon'
  package_name = 'python36'
else
  package_name = 'python3'
end

python_runtime '3' do
  provider :system

  # See https://github.com/poise/poise-python/issues/107
  pip_version '9.0.3'

  options :system, package_name: package_name
end

Any word on this getting fixed and merged into master. This breaks the AWS code deploy agent which takes down our entire environment. We use berks to manage dependencies and I haven't found a way to overwrite the pip version from an attributes wrapper.

also running into this issue

same issue here. this regex is super weird, what is it even supposed to do? also, you don't need to escape that backslash there, you're using a raw string literal...

Literally everyone is running into this issue, please stop with the me toos. It takes a few days to test with new major releases and whatnot, for now you’ll need to pin back to pip 9 to use this cookbook.

@coderanger,
Is there a way to pin pip 9 through an attribute?
I see override['poise-python']['options']['pip_version'] = false can I use my wrapper to pin from there?

Yes, you can set any provider option via that system.

Serious question, what on earth was that regex trying to match?

To get around this issue I had to specify a different pip_url

this is a major issue.    Is this cookbook no longer maintained?

@davidmontgom I already responded to you once. Stop it.

@coderanger Needed a bit of guidance since I'm pretty new to chef. Went through the concept of wrapper cookbooks. From the comment, all I need to do is:

Include dependency in metadata.rb
In my cookbook's default.rb, include_recipe 'poise-python::default'
Include the line below line in my cookbook's attribute file
`default['poise-python']['options']['get_pip_url'] = 'https://raw.githubusercontent.com/pypa/get-pip/master/2.6/get-pip.py'``

Could you confirm?

oh cool. Thanks. Will implement that till this gets fixed.

@coderanger thanks for the info, might I suggest to make the documentation for the pip_version attribute a bit more clear?
pip_version – Version of pip to install. If set to true, use the latest. If set to false, do not install pip. For backward compatibility, can also be set to a URL instead of get_pip_url. (default: true)

It makes sense that it takes a version string since it says: Version of pip to install, but I find the second sentence to be a bit misleading in that it makes me think that this attribute should be a boolean.

I'm having a little trouble understanding the lack of sense of urgency here.  Many (all) people are affected by this who updated to pip 10.  Yet, it's been over two weeks and no fix is in place nor has the PR which could resolve this been merged.
I get that everyone is busy and what not, but @coderanger you've replied several times to this thread indicating you have at least some availability.  I understand wanting to test and what not, but at this point what are you worried about?  Making the cookbook more broken?  This is hotfix territory.  Push out a fix that hopefully works and worry about thoroughly testing and/or refining it later.

@et304383,
There is urgency here but @coderanger is only one person. The version check isn't the only issue with PIP 10 they changed some of the API calls. You can pin PIP to 9.0.3 to get it running again. If you use a wrapper like I do because I'm really only using this cookbook for the aws-codedeploy-agent you can use the override attribute stated above. If you are actually using the cookbook each item has the ability to set the PIP version so pin it there.

@barracksbuilder @et304383 Open source software!
Im attempting to use the attribute work around on CentOS 6.9 though and running into

For others here I came across this issue via the cloudcli chef cookbook which depends on this cookbook (both at latest versions). Adding the following to the attributes file of my wrapper cookbook did the trick.
override['poise-python']['options']['pip_version'] = '9.0.3'

In case anyone is using the get-pip.py from pypa.io, here’s how I dealt with that:
remote_file file_cache_path('get-pip.py') do
  source 'https://bootstrap.pypa.io/get-pip.py'
  notifies :run, "python_execute[#{file_cache_path('get-pip.py')} 'pip<10']", :immediately
end

python_execute "#{file_cache_path('get-pip.py')} 'pip<10'" do
  action :nothing
end

@jeffbyrnes I'm confused in what order do you run this?
I also tried to set pip_version to false in the python_runtime resource but no dice.
python_execute requires a runtime, so not sure how you managed to run it? Or maybe you have a system python package installed before?

Poise-python 1.7.0 is up with full support for both Pip 10 and Chef 14.


Insertions after the stop codon annotated as plain Insertions

Testcase using genome GRCm38, over transcript ENSMUST00000086738
##fileformat=VCFv4.1
#CHROM	POS	ID	REF	ALT	QUAL	FILTER	INFO	FORMAT

# Insertion immediately after the stop codon
# TGA ACT ATT -> TAG CTT GGA ACT ATT...
chr1	100484701	.	A	ACAGCAG	5000	.	.	.

This is annotated as
Insertion(variant=chr1 g.100484701_100484702insCAGCAG, transcript_name=Cntnap5b-001, transcript_id=ENSMUST00000086738, effect_description=p.1292insQQ)
* len(original_protein_sequence) = 1292
* aa_ref = ""
* aa_alt = "QQ"
* aa_mutation_start_offset = 1292

which an insertion on the stop codon position, pushing it downstream.
If https://github.com/hammerlab/varcode/issues/167 is fixed, then the annotation becomes
Insertion(variant=chr1 g.100484701_100484702insCAGCAG, transcript_name=Cntnap5b-001, transcript_id=ENSMUST00000086738, effect_description=p.1293insQQ)
* len(original_protein_sequence) = 1292
* aa_ref = ""
* aa_alt = "QQ"
* aa_mutation_start_offset = 1293

which is an insertion after the stop codon.

Suggested fix, also fixes https://github.com/hammerlab/varcode/issues/168
--- a/varcode/coding_effect.py  2016-08-27 15:08:24.011702562 +0200
+++ b/varcode/coding_effect.py  2016-08-28 23:36:50.017038233 +0200
@@ -108,8 +108,13 @@

     sequence_from_start_codon = str(sequence[cds_start_offset:])

-    # is this an in-frame mutations?
-    if (len(ref) - len(alt)) % 3 == 0:
+    # Offset of first unmodified codon after the mutation.
+    transcript_ref_end_offset = ((transcript_offset + len(ref) + 2) // 3) * 3
+
+    # Is this an in-frame mutation which does not shift the codons downstream?
+    if ((len(ref) - len(alt)) % 3 == 0 and
+        transcript_offset < cds_stop_offset and
+        transcript_ref_end_offset <= cds_stop_offset):
         return in_frame_coding_effect(
             ref=ref,
             alt=alt,
--- a/varcode/frameshift_coding_effect.py       2016-08-28 23:39:41.848240000 +0200
+++ b/varcode/frameshift_coding_effect.py       2016-08-29 01:48:33.711213476 +0200
@@ -68,11 +68,12 @@
         # TODO: scan through sequence_from_mutated_codon for
         # Kozak sequence + start codon to choose the new start
         return StartLoss(variant=variant, transcript=transcript)
-    elif mutated_codon_index == original_protein_length:
-        return StopLoss(
+    elif mutated_codon_index > original_protein_length:
+        return Silent(
             variant=variant,
             transcript=transcript,
-            extended_protein_sequence=mutant_protein_suffix)
+            aa_pos=mutated_codon_index,
+            aa_ref="")

     # the frameshifted sequence may contain some amino acids which are
     # the same as the original protein!
@@ -81,12 +82,15 @@
         alt=mutant_protein_suffix)
     n_unchanged_amino_acids = len(unchanged_amino_acids)
     mutation_start_position = mutated_codon_index + n_unchanged_amino_acids
-    if mutation_start_position >= original_protein_length:
+
+    assert mutation_start_position <= original_protein_length
+
+    if mutation_start_position == original_protein_length:
         # frameshift is either extending the protein or leaving it unchanged
         if len(mutant_protein_suffix) == 0:
             # miraculously, this frameshift left the protein unchanged,
             # most likely by turning one stop codon into another stop codon
-            aa_ref = original_protein_sequence[mutated_codon_index]
+            aa_ref = original_protein_sequence[mutated_codon_index:] or "*"
             return Silent(
                 variant=variant,
                 transcript=transcript,

Fixed by https://github.com/hammerlab/varcode/pull/173


Stacked relative

I would like to get an effect of the stacked demo, where multiple sticky headers politely let one another to be at the top of the screen, only relative to an arbitrary container (relative option in the library).
It seems at the moment this is not supported.

I am facing the same issue. Were you able to fix it?

No. But depending on your browser support, position: sticky might be good enough now or in the future.

@denis-sokolov I'm not quite understanding what the problem is. You're trying to replicate the stacked demo?

@vcarl, yes, the stacked demo, but not at the top of the screen, but at the top of an arbitrary container, like in the “relative” demo.

Gotcha, okay. It definitely does seem like if the stacked demo works, and the relative demo works, a stacked relative demo should work as well.
I'm still familiarizing myself with the library, would you be able to make a "stacked relative" demo as a PR? It seems like a valuable addition to the docs, and if it produces buggy behavior it would help me a lot when I go to debug it.

Sorry, it’s been a while, I’m not using it in my current project and it’ll cost me much more effort to get back into the context.

Okay, that's fine! I'll leave this open as a doc enhancement issue, and will open a bug issue if I find something's not working right.


Intermittent flickering on Fibaro FGD212

Is your problem within Home Assistant (Core or Z-Wave JS Integration)?
NO, my problem is NOT within Home Assistant or the ZWave JS integration
Is your problem within Z-Wave JS UI (formerly ZwaveJS2MQTT)?
NO, my problem is NOT within Z-Wave JS UI
Checklist

[X] I have checked the troubleshooting section and my problem is not described there.

[X] I have read the changelog and my problem was not mentioned there.

Describe the bug
What causes the bug?
I'm not really sure, it's intermittent :(
What do you observe?
The light switch on and off every 4-5 seconds during ~1 minute, then switch off
What did you expect to happen?
The ligth stays on
Steps to reproduce the behavior:
Light on the lights using the wall switch
Device information
Manufacturer: Fibargroup
Model name: FGD212
Firmware: 3.3
Node ID in your network: 4
How are you using node-zwave-js?

[ ] zwave-js-ui (formerly zwavejs2mqtt) Docker image (latest)
[ ] zwave-js-ui (formerly zwavejs2mqtt) Docker image (dev)
[ ] zwave-js-ui (formerly zwavejs2mqtt) Docker manually built (please specify branches)
[ ] ioBroker.zwave2 adapter (please specify version)
[X] HomeAssistant zwave_js integration  (please specify version)
[ ] pkg
[ ] node-red-contrib-zwave-js (please specify version, double click node to find out)
[ ] Manually built from GitHub (please specify branch)
[ ] Other (please describe)

Which branches or versions?
home assistant hassio addon version:  0.1.74
Did you change anything?
no
If yes, what did you change?

Did this work before?
Yes (please describe)
If yes, where did it work?
I have 2 other modules, and I don't have this behavior on them.
Attach Driver Logfile
When this appears, I got a bunch of
2022-11-11T17:08:51.492Z CNTRLR   [Node 004] [Notification]
                                    type:  Power Management
                                    event: Voltage drop/drift
2022-11-11T17:08:59.467Z CNTRLR   [Node 004] [Notification]
                                    type:  Power Management
                                    event: Voltage drop/drift
2022-11-11T17:09:12.473Z CNTRLR   [Node 004] [Notification]
                                    type:  Power Management
                                    event: Voltage drop/drift
2022-11-11T17:09:21.445Z CNTRLR   [Node 004] [Notification]
                                    type:  Power Management
                                    event: Voltage drop/drift

Also, on home assistant, I have a diagnostic panel (see screenshot) that display an "Unsafe" message on Power Management - Load error.
I have another FGD212 which is in the same state (unsafe) but the light works well.
My next step is to switch 2 of my FGD212 to see if this one is not faulty.

This looks like an issue with the switch's power supply. It even tells you that there is a voltage drop.

I tried to replace a bulb with a non-flickering one, but the problem persist.
I'll try to switch with another FGD212 to see if it's the module that is dying


FHEM restart causes smartnames to be forgotten

Everytime I restart fhem, I see in the cloud adapter that my fhem devices are not there anymore.
As I understand adding the devices in the cloud adapter (I use iot) the smartnames gets saved along with the objects:
 "fhem.0.Shelly1_baie.state": {     "_id": "fhem.0.Shelly1_baie.state",     "type": "state",     "common": {       "name": "Shelly1_baie state",       "type": "string",       "read": true,       "write": true,       "role": "state",       "smartName": {         "de": "Bad",         "smartType": "LIGHT"       }     },     "native": {       "Name": "Shelly1_baie",       "Attribute": "state",       "Readings": true,       "onoff": true,       "ts":<PHONE_NUMBER>678     },     "from": "system.adapter.fhem.0",     "ts":<PHONE_NUMBER>451,     "acl": {       "object": 1636,       "state": 1636     }   },
the same device after fhem restarted:
"fhem.0.Shelly1_baie.state": {     "_id": "fhem.0.Shelly1_baie.state",     "type": "state",     "common": {       "name": "Shelly1_baie state",       "type": "string",       "read": true,       "write": true,       "role": "state"     },     "native": {       "Name": "Shelly1_baie",       "Attribute": "state",       "Readings": true,       "onoff": true,       "ts":<PHONE_NUMBER>356     },     "from": "system.adapter.fhem.0",     "ts":<PHONE_NUMBER>118,     "acl": {       "object": 1636,       "state": 1636     }   },
Any way to prevent that?

Which fhem version?

Latest Revision: 18358
File                Rev   Last Change
fhem.pl             18343 2019-01-20
I managed to get lights to work by adding an alias to the name (now when the adapter finds them it registers the Smartname with the alias name, but doesn't set type to light - any idea how to do that)?
But it still doesn't work for rollos: there I had to make some changes (add min0, max100, type=number), and afterwards add the smartname

I mean which Adapter version? :-)

1.2.1 installed from github

check fhem.0.info.Configurations.autoSmartName
false= no update smartName restart
true = update smartName every restart


Develop upstream sync 180730

Weekly sync up, submit to check with CI.

@sunway513 in your latest commit you added one missing file. in order to retain the history of that particular file, I'm wondering would it be possible to revise this PR so the file is not deleted in the first place?

@whchung done, please review.

@deven-amd , could you rebase your commit with c7366c6 and submit your changes with -f ?

@sunway513 / @deven-amd please help assess if the 2 failed tests:
//tensorflow/python/distribute:distribute_coordinator_test
//tensorflow/python/keras:model_subclassing_test

are newly added or not, and edit CI whitelist accordingly. thx.

//tensorflow/python/feature_column:feature_column_v2_test ?

@sunway513 / @deven-amd seems //tensorflow/python/estimator:dnn_linear_combined_test  is kind of flaky. maybe whitelist it? it seems to be related to rocPRIM issue @deven-amd raised yesterday at https://github.com/ROCmSoftwarePlatform/rocPRIM/issues/18

@whchung , done. Okay to merge or you want another CI check?

let's give CI another spin just to make sure then we'll take it. really want no surprises in develop-upstream branch

@deven-amd / @sunway513 I merged this PR even though there is still one failure on ROCm path. Please help check. Build log is at: http://<IP_ADDRESS>:21096/job/tensorflow-upstream-unit-tests/253/

@whchung I think we missed the steps to add tags before/after merging to develop-upstream branch.
In case this PR cause further issues, we need the following command to revert:
git checkout develop-upstream && git revert -m 1 c505257

@sunway513 please help apply tags on develop-upstream.

@whchung any idea to make up the "pre merge" tag?

Thanks for the tricks @whchung , the flags are up to date.


Update startupposition-property.md

Remove extra space

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

VBA/Language-Reference-VBA/articles/startupposition-property.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

VBA/Language-Reference-VBA/articles/startupposition-property.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

All content in this repository has been migrated to a new repository (https://github.com/MicrosoftDocs/VBA-Docs). The changes in this PR have been applied in the new repository, so this PR is being closed. The changes will be published to the live docs soon. Thank you for your contribution!


changeset for mime-types

Current dependencies on/for this PR:

main

PR #448   👈

This stack of pull requests is managed by Graphite.


merge latest developments for release

Doneness:

[x] Testing
[x] Documentation
[x] package.json version update
[x] rimble.consensys.design version update
[ ] storybook documentation update

@MikeLockz can you review?

closes #146


Issue with /docs/getting-started-guides/cloudstack/

This is a...

[x ] Bug Report

Problem:
The documentation regarding k8s installation on CloudStack is not up to date. The Ansible playbook refer to a github repository now defined to deploy on AWS (see here). All previous job for CS has been deleted.
Proposed Solution:
We can retrieve from git history the original work for CS (last related commit). So I havae forked this repo to an kubernetes-cloudstack repo and update it. In the meantime we should correct the k8s documentation related page. I begin to work on it there and I can validate it works.
Page to Update:
https://kubernetes.io/docs/getting-started-guides/cloudstack/
Additional Information.
I would be happy to update the k8s website page too and created the following issue https://github.com/kubernetes/kubernetes.github.io/pull/3430.

Thanks for the update.  Closing this issue.


Map is broken

Seems like mapbox now requires a payment option even for the free tier

openstreetmap tiles seem to work.
Geocoding requires mapbox though, so it will remain broken


MIC-557

Codecov Report
Attention: Patch coverage is 85.18519% with 4 lines in your changes missing coverage. Please review.

Project coverage is 90.95%. Comparing base (afce187) to head (e1b9c2c).

Files with missing lines
Patch %
Lines

...engesupportinterventionplanapi/model/PlanDetail.kt
77.77%
0 Missing and 2 partials :warning:

...rtinterventionplanapi/model/request/PlanRequest.kt
33.33%
2 Missing :warning:

Additional details and impacted files
@@             Coverage Diff              @@
##               main     #154      +/-   ##
============================================
+ Coverage     90.82%   90.95%   +0.12%
- Complexity      746      747       +1
============================================
  Files           137      138       +1
  Lines          2148     2155       +7
  Branches        143      146       +3
============================================
+ Hits           1951     1960       +9
+ Misses          132      130       -2
  Partials         65       65

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Empty I/O dashboard from last upgrade

Since upgrade from 3.0.1 to 3.0.2 in Disk Activity from Scylla Per-Server Disk I/O (master & 3.0 version)
All metrics values disappeared

All those metrics come from node_exporter, could it be that you didn't start it for some reason?

Sorry @amnonh
i see node_exporter service on systemd
flag provided but not defined: -collector.interrupts
i will report in scylla project

if that is the case, it means you are running node_exporter 0.14 instead of
0.17
On Tue, Feb 5, 2019 at 11:37 PM fggarcia<EMAIL_ADDRESS>wrote:

Closed #492
https://github.com/scylladb/scylla-grafana-monitoring/issues/492.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/scylladb/scylla-grafana-monitoring/issues/492#event-2119726525,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ACBRv504uOJYvHLH97M8kRgBLzNR_9dcks5vKfmJgaJpZM4akA79
.

Hi @amnonh
Now that i upgrade node_exporter to 0.17
node_exporter --version
node_exporter, version 0.17.0 (branch: HEAD, revision: f6f6194a436b9a63d0439abc585c76b19a206b21)
  build user:       root@322511e06ced
  build date:       20181130-15:51:33
  go version:       go1.11.2

I saw that you use node_disk_bytes_read to extract disk name.....but:
old instance (0.14 node_exporter)
node_exporter --version
node_exporter, version 0.14.0 (branch: master, revision: 840ba5dcc71a084a3bc63cb6063003c1f94435a6)
  build user:       root@bb6d0678e7f3
  build date:       20170321-12:12:54
  go version:       go1.7.5

and extract metrics:
curl -X GET 'http://localhost:9100/metrics' | grep 'node_disk_bytes_read'
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0# HELP node_disk_bytes_read The total number of bytes read successfully.
# TYPE node_disk_bytes_read counter
node_disk_bytes_read{device="nvme0n1"} 7.327791104e+09
node_disk_bytes_read{device="nvme1n1"} 1.09974226105344e+14
100 95153  100 95153    0     0  14.3M      0 --:--:-- --:--:-- --:--:-- 15.1M

instance with new version of node_exporter:
node_exporter --version
node_exporter, version 0.17.0 (branch: HEAD, revision: f6f6194a436b9a63d0439abc585c76b19a206b21)
  build user:       root@322511e06ced
  build date:       20181130-15:51:33
  go version:       go1.11.2

curl -X GET 'http://localhost:9100/metrics' | grep 'node_disk_bytes_read'
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 55886    0 55886    0     0  4168k      0 --:--:-- --:--:-- --:--:-- 4548k

You are right, to be backwards compatible we configure prometheus to change
the metrics nanes so the dashboards would continue to work.
You can learn about it more in the prometheus configuration part here:
https://github.com/scylladb/scylla-grafana-monitoring/blob/master/README.md
On Friday, February 8, 2019, fggarcia<EMAIL_ADDRESS>wrote:

Hi @amnonh https://github.com/amnonh
Now that i upgrade node_exporter to 0.17
node_exporter --version
node_exporter, version 0.17.0 (branch: HEAD, revision: f6f6194a436b9a63d0439abc585c76b19a206b21)
build user:       root@322511e06ced
build date:       20181130-15:51:33
go version:       go1.11.2
[image: image]
https://user-images.githubusercontent.com/1306596/52458385-d058f100-2b3d-11e9-8dad-b23a53c9e753.png
I saw that you use node_disk_bytes_read to extract disk name.....but:
old instance (0.14 node_exporter)
node_exporter --version
node_exporter, version 0.14.0 (branch: master, revision: 840ba5dcc71a084a3bc63cb6063003c1f94435a6)
build user:       root@bb6d0678e7f3
build date:       20170321-12:12:54
go version:       go1.7.5
and extract metrics:
curl -X GET 'http://localhost:9100/metrics' | grep 'node_disk_bytes_read'
% Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
Dload  Upload   Total   Spent    Left  Speed
0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0# HELP node_disk_bytes_read The total number of bytes read successfully.
TYPE node_disk_bytes_read counter
node_disk_bytes_read{device="nvme0n1"} 7.327791104e+09
node_disk_bytes_read{device="nvme1n1"} 1.09974226105344e+14
100 95153  100 95153    0     0  14.3M      0 --:--:-- --:--:-- --:--:-- 15.1M
instance with new version of node_exporter:
node_exporter --version
node_exporter, version 0.17.0 (branch: HEAD, revision: f6f6194a436b9a63d0439abc585c76b19a206b21)
build user:       root@322511e06ced
build date:       20181130-15:51:33
go version:       go1.11.2
curl -X GET 'http://localhost:9100/metrics' | grep 'node_disk_bytes_read'
% Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
Dload  Upload   Total   Spent    Left  Speed
100 55886    0 55886    0     0  4168k      0 --:--:-- --:--:-- --:--:-- 4548k
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/scylladb/scylla-grafana-monitoring/issues/492#issuecomment-461684354,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ACBRv2nOTFwGv5btfPGGmqee81So1-8Yks5vLPi1gaJpZM4akA79
.

Hi @amnonh
you are mention to section "Using your own Prometheus server node_exporter specifc configuration"
where should this configuration stay? On scylla server or scylla-grafana-monitoring server(we use docker image)?
If this configuration should stay in scylla-server, why do scylla install not create this file?
Thanks you for your help

It's part of prometheus configuration, part of the manager configuration.
If you are using the docker solution, it will be configure for you, so you
don't need to do anything
On Friday, February 8, 2019, fggarcia<EMAIL_ADDRESS>wrote:

Hi @amnonh https://github.com/amnonh
you are mention to section "Using your own Prometheus server node_exporter
specifc configuration"
where should this configuration stay? On scylla server or
scylla-grafana-monitoring server(we use docker image)?
If this configuration should stay in scylla-server, why do scylla install
not create this file?
Thanks you for your help
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/scylladb/scylla-grafana-monitoring/issues/492#issuecomment-461816832,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ACBRv6XpSW0zRomQsWQyMWVotiiofzUFks5vLYczgaJpZM4akA79
.

Yes, sorry it was a stupid question :P
I find that configuration in prometheus.yml.template
I upgrade to node_exporter 0.17, but i still don't see that metrics in Scylla-Per-Server Disk

For example, our disk name is vda. Not exist in disk label options

@fggarcia First I'm happy that it's working better already.
Second, to understand the source of the problem why you don't see the relevant label, can you please open the prometheus dashboard:
http://{ip}:9090 (for example if you are checking it locally it will be http://<IP_ADDRESS>:9090)
And check what the value of:
node_disk_bytes_read
It should return all values from all hosts.

There are only metrics from instances was not upgraded to node-exporter 0.17

it's either that prometheus cannot read from those nodes, or the name change in the configuration is not working propertly.
To check the first option check the targets tab in prometheus:
http://{ip}:9090/targets (where ip is the ip address).
make sure that all the node_exporter are up, if they are not, check what is the error code.
also, make sure that you are running the latest monitoring version.

Migrate all servers to node-exporter 0.17

But already happen that not show disks monitor

update:
docker exec -it aprom cat /etc/prometheus/prometheus.yml
global:
  scrape_interval: 5s # By default, scrape targets every 5 second.
  scrape_timeout: 4s # Timeout before trying to scape a target again

  # Attach these labels to any time series or alerts when communicating with
  # external systems (federation, remote storage, Alertmanager).
  external_labels:
    monitor: 'scylla-monitor'
rule_files:
  - /etc/prometheus/prometheus.rules.yml
    #
# Alerting specifies settings related to the Alertmanager.
alerting:
        #  alert_relabel_configs:
        #    [ - <relabel_config> ... ]
  alertmanagers:
  - static_configs:
    - targets:
        - <IP_ADDRESS>:9093

scrape_configs:
- job_name: scylla
  honor_labels: false
  file_sd_configs:
    - files:
      - /etc/scylla.d/prometheus/scylla_servers.yml
  relabel_configs:
    - source_labels: [__address__]
      regex:  '(.*):.+'
      target_label: instance
      replacement: '${1}'

- job_name: node_exporter
  honor_labels: false
  file_sd_configs:
    - files:
      - /etc/scylla.d/prometheus/node_exporter_servers.yml
  relabel_configs:
    - source_labels: [__address__]
      regex:  '(.*):\d+'
      target_label: instance
      replacement: '${1}'
    - source_labels: [__address__]
      regex:  '(.*):\d+'
      target_label: __address__
      replacement: '${1}:9100'
    - source_labels: [__name__]
      regex:  'node_disk_read_bytes_total'
      target_label: __name__
      replacement: 'node_disk_bytes_read'
    - source_labels: [__name__]
      regex:  'node_disk_written_bytes_total'
      target_label: __name__
      replacement: 'node_disk_bytes_written'
    - source_labels: [__name__]
      regex:  'node_disk_reads_completed_total'
      target_label: __name__
      replacement: 'node_disk_reads_completed'
    - source_labels: [__name__]
      regex:  'node_disk_writes_completed_total'
      target_label: __name__
      replacement: 'node_disk_writes_completed'
    - source_labels: [__name__]
      regex:  'node_filesystem_avail_bytes'
      target_label: __name__
      replacement: 'node_filesystem_avail'
    - source_labels: [__name__]
      regex:  'node_network_receive_bytes_total'
      target_label: __name__
      replacement: 'node_network_receive_bytes'
    - source_labels: [__name__]
      regex:  'node_network_receive_packets_total'
      target_label: __name__
      replacement: 'node_network_receive_packets'
    - source_labels: [__name__]
      regex:  'node_network_transmit_bytes_total'
      target_label: __name__
      replacement: 'node_network_transmit_bytes'
    - source_labels: [__name__]
      regex:  'node_network_transmit_packets_total'
      target_label: __name__
      replacement: 'node_network_transmit_packets'
    - source_labels: [__name__]
      regex:  'node_filesystem_size_bytes'
      target_label: __name__
      replacement: 'node_filesystem_size'

- job_name: scylla_manager
  honor_labels: false
  file_sd_configs:
    - files:
      - /etc/scylla.d/prometheus/scylla_manager_servers.yml
  metric_relabel_configs:
    - source_labels: [host]
      target_label: instance

- job_name: 'prometheus'
  # Override the global default and scrape targets from this job every 5 seconds.
  scrape_interval: 5s
  static_configs:
    - targets:
      - localhost:9090

For example:
node_disk_read_bytes_total

but
node_disk_bytes_read

I don't know why the job not replace the original metrics.
If i replace manually metrics in dashboard, the dashboard works correctly.
Any idea?
PD: I delete/remove original images, and download docker images again.

See #496

@amnonh Thanks you so much. I pull the last commit to our monitoring server and it's all right now.