'preserveTags' flag for divideSeries, divideSeriesLists, asPercent

Is your feature request related to a problem? Please describe.
Several functions (at least divideSeries, divideSeriesLists, and asPercent) trash the tags of the inputs. This makes it difficult to use features in grafana that require tags (e.g. "Labels to Fields" transformation) or other functions like groupByTags post division.
It would be nice to have an option to preserve tags where the dividend and divisor have the same values.
Describe the solution you'd like
Add a flag (default to false for backwards compatibility) to preserve overlapping tags
Describe alternatives you've considered
N/A

Perhaps even easier (more useful?) an option to keep the tags of the dividend for these functions, rather than strictly overlapping tags.

Hi @shanson7,
Sorry, but could you please elaborate your request a bit? What do you mean "trash the tags of the inputs" ?

What do you mean "trash the tags of the inputs" ?

Tags go in but they don't come out.

Maybe add some useful example which can be used in tests?

Sure, what format?

Tags go in but they don't come out.

Sorry, I still not sure that I understand what that means. I do not use tags in production, though.

Sure, what format?

Example input, current output, desired output ?

Example input, current output, desired output ?

Haha, yeah I meant what format do tests take for series/tags. I'll dig it out of the code.

Ok so let's say we have 2 input series representing the number of requests and the total time taken (divisor and dividend respectively):
divisor.name = 'requests.count'
divisor.tags = {
  'host' : 'h1',
  'instanceName': 'instance1'
}

dividend.name = 'requests.seconds'
dividend.tags = {
  'host' : 'h1',
  'instanceName': 'instance1'
}

Currently, if we do divideSeries(dividend, divisor) we get a new series:
result.name = 'divideSeries(requests.seconds,requests.count)'
result.tags = {}

It would be desirable to maintain the tags like:
result.name = 'divideSeries(requests.seconds,requests.count)'
result.tags = {
  'host' : 'h1',
  'instanceName': 'instance1'
}

You can expand this to divideSeriesLists where losing the tags makes the output series indistiguishable and we need to rely on functions like aliasByTags before the division to make it work. However, grafana is becoming more and more tag aware and features that rely on tags become inaccessible.

imho the fact that the resulting series have no tags should be considered a bug, and I agree that we should propagate tags in roughly the same way as we do for other aggregation functions.  My suggestion would be that by default all tags of the dividend series should be propagated, along with a new tag to be consistent with the behavior of other aggregation-type functions.  We may want to consider also propagating tags on the divisor series that are not present on the dividend, whether by default or with an option flag.  Does that seem reasonable?

My suggestion would be that by default all tags of the dividend series should be propagated, along with a new tag to be consistent with the behavior of other aggregation-type functions. We may want to consider also propagating tags on the divisor series that are not present on the dividend, whether by default or with an option flag. Does that seem reasonable?

I get a little nervous about changing this type of behavior after the fact (bug or not) because some people might be actively working around it and it could break existing dashboards. However, the behavior you describe does sound reasonable to me (backward compatibility aside).

imho the fact that the resulting series have no tags should be considered a bug, and I agree that we should propagate tags in roughly the same way as we do for other aggregation functions.

+1. I'm not too worried about accommodating people who may have worked around this lack of tags, as it is such as obvious bug; also tags are not commonly used yet. But ultimately this is a question for the graphite maintainers (which i'm not part of).
Note the asPercent signature: asPercent(seriesList, total=None, *nodes)
since *nodes swallows all remaining args, i don't think we can just add an arg at the end, we would probably add it in the middle (before or after total), because it would be an optional flag, this should be fine (in both graphite and metrictank)
Note that Dan suggests setting a new tag aggregatedBy: <func-name> as well as either of these:

all dividend tags only
all dividend tags + tags that are set only on divisor (this is effectively the union)

In that proposal, it is implied that if the same tag keys are set between dividend and divisor, but with different values, dividend wins.  Keeping a tag value of one of the series as is, when the other series directly contradicts is, seems like a bad idea....
E.g. imagine a division between:
http_responses{service=A,code=200}

and
http_responses{service=A,code=total}

we have the choice between:

let dividend or divisor overrule the other. per Dan (service=A,code=200)
drop tag (service=A)
generate a new tag that's the unison of them (service=A,code=200/total)

2 is probably not that useful (we may drop precisely what distinguishes different series), Dan's idea is actually not so bad (we typically look at a series "with respect to" another, where the "another" is implied and keeping only the former explicit may be good enough.), the 3rd one seems most factually correct because it acknowledges the contribution of both components of the division. but the naive formatting may be hard to decipher if either of the tag values already contain a slash.
Anyway, that tangent aside, I note that there's two more options to consider to dan's proposal:

intersection between dividend and divisor (key-wise intersection that is, value is subject to the same question above)
divisor tags only

I agree the latter isn't all that useful e.g. in asPercent() when we have multiple distinct dividends and a single divisor, we would want to distinguish based on divident tags; whereas in other cases where we map dividends and divisors 1:1 (this happens in divideSeries, divideSeriesList and certain asPercent invocations), I think "divisor tags only" is just as defensible as "dividend tags only".
Thus I think all these modes make sense:

dividend only
divisor only
union
intersection

and then i think it would make sense to have an additional flag to control the behavior in the latter 2 cases, in case of tag-matches with value mismatches:

drop
merge
divisor wins
dividend wins

Agreed with @DanCech here - tags should not be swallowed by default.
Not sure do we need to implement full spectrum of modes/flags, though. But I think we need listen what users of tags functionality  want - otherwise indeed implement full fledged solution as mostly flexible, as @Dieterbe described.

We have been using tags for a while now, so I expect many dashboards currently "work" with the existing behavior. I expect many of those will be negatively impacted by a backwards incompatible change (not to mention any code that is making queries using these functions).
That being said, I'm not strictly against a backwards incompatible change, it just shouldn't be done lightly. The issue might be significantly more prevalent if you expand it to all functions that aggregate/combine multiple series, e.g multiplySeries or even sum/avg/etc.

We plan to provide a concrete proposal for this feature after the holidays

Looking on how to expand these function signatures is interesting...
first of all, looking at the function signature we have today for asPercent, it seems to me that even when trying to not use total, the first node specified will always be assigned to total.  See below. am i missing something? how does this work today?
$ cat stock.py
def asPercent(seriesList, total=None, *nodes):
    print ("series", seriesList)
    print ("total", total)
    print ("nodes", nodes)
asPercent("seriesList", 1, 2, 3)
$ python2 stock.py
('series', 'seriesList')
('total', 1)
('nodes', (2, 3))
$ python3 stock.py
series seriesList
total 1
nodes (2, 3)

that aside, trying to add the arg at the end, only works in python3 (invalid syntax in py2)
$ cat newargend.py
def asPercent(seriesList, total=None, *nodes, preserveTags=False):
    print ("series", seriesList)
    print ("total", total)
    print ("nodes", nodes)
    print("preserveTags", preserveTags)
asPercent("seriesList", "n1", "n2", "n3", preserveTags=True)
$ python2 newargend.py
  File "newargend.py", line 3
    def asPercent(seriesList, total=None, *nodes, preserveTags=False):
                                                             ^
SyntaxError: invalid syntax
$ python3 newargend.py
series seriesList
total n1
nodes ('n2', 'n3')
preserveTags True

adding the new arg as keyword arg before nodes and trying to set it as kwarg, does not work:
$ cat newargmid.py
def asPercent(seriesList, total=None, preserveTags=False, *nodes):
    print ("series", seriesList)
    print ("total", total)
    print("preserveTags", preserveTags)
    print ("nodes", nodes)
asPercent("seriesList", preserveTags=True, 1, 2, 3)
$ python2 newargmid.py
  File "newargmid.py", line 6
    asPercent("seriesList", preserveTags=True, 1, 2, 3)
SyntaxError: non-keyword arg after keyword arg
$ python3 newargmid.py
  File "/home/dieter/code/sandbox/graphite-preservetags/newargmid.py", line 6
    asPercent("seriesList", preserveTags=True, 1, 2, 3)
                                                      ^
SyntaxError: positional argument follows keyword argument

but trying to call it not specifying it as an arg suffers from the same problem as with total (unless I'm missing something)
$ cat newargmid.py
def asPercent(seriesList, total=None, preserveTags=False, *nodes):
    print ("series", seriesList)
    print ("total", total)
    print("preserveTags", preserveTags)
    print ("nodes", nodes)
asPercent("seriesList", 1, 2, 3)
$ python2 newargmid.py
('series', 'seriesList')
('total', 1)
('preserveTags', 2)
('nodes', (3,))
$ python3 newargmid.py
series seriesList
total 1
preserveTags 2
nodes (3,)
$

So the only solution I see is adding it at the end and dropping support for python2. but that seems a bit much to ask.

Looking on how to expand these function signatures is interesting...
first of all, looking at the function signature we have today for asPercent, it seems to me that even when trying to not use total, the first node specified will always be assigned to total.  See below. am i missing something? how does this work today?
$ cat stock.py
def asPercent(seriesList, total=None, *nodes):
    print ("series", seriesList)
    print ("total", total)
    print ("nodes", nodes)
asPercent("seriesList", 1, 2, 3)
$ python2 stock.py
('series', 'seriesList')
('total', 1)
('nodes', (2, 3))
$ python3 stock.py
series seriesList
total 1
nodes (2, 3)

that aside, trying to add the arg at the end, only works in python3 (invalid syntax in py2)
$ cat newargend.py
def asPercent(seriesList, total=None, *nodes, preserveTags=False):
    print ("series", seriesList)
    print ("total", total)
    print ("nodes", nodes)
    print("preserveTags", preserveTags)
asPercent("seriesList", "n1", "n2", "n3", preserveTags=True)
$ python2 newargend.py
  File "newargend.py", line 3
    def asPercent(seriesList, total=None, *nodes, preserveTags=False):
                                                             ^
SyntaxError: invalid syntax
$ python3 newargend.py
series seriesList
total n1
nodes ('n2', 'n3')
preserveTags True

adding the new arg as keyword arg before nodes and trying to set it as kwarg, does not work:
$ cat newargmid.py
def asPercent(seriesList, total=None, preserveTags=False, *nodes):
    print ("series", seriesList)
    print ("total", total)
    print("preserveTags", preserveTags)
    print ("nodes", nodes)
asPercent("seriesList", preserveTags=True, 1, 2, 3)
$ python2 newargmid.py
  File "newargmid.py", line 6
    asPercent("seriesList", preserveTags=True, 1, 2, 3)
SyntaxError: non-keyword arg after keyword arg
$ python3 newargmid.py
  File "/home/dieter/code/sandbox/graphite-preservetags/newargmid.py", line 6
    asPercent("seriesList", preserveTags=True, 1, 2, 3)
                                                      ^
SyntaxError: positional argument follows keyword argument

but trying to call it not specifying it as an arg suffers from the same problem as with total (unless I'm missing something)
$ cat newargmid.py
def asPercent(seriesList, total=None, preserveTags=False, *nodes):
    print ("series", seriesList)
    print ("total", total)
    print("preserveTags", preserveTags)
    print ("nodes", nodes)
asPercent("seriesList", 1, 2, 3)
$ python2 newargmid.py
('series', 'seriesList')
('total', 1)
('preserveTags', 2)
('nodes', (3,))
$ python3 newargmid.py
series seriesList
total 1
preserveTags 2
nodes (3,)
$

So the only solution I see is adding it at the end and dropping support for python2. but that seems a bit much to ask.

@Dieterbe: IMO it's easier to introduce new fuction then, with different arguments.

@Dieterbe: IMO it's easier to introduce new fuction then, with different arguments.

Gave this some more thought and discussed a bit internally with some set theory nerds at Grafana ( :) )
First of all, I think figuring out which tags to set on the results may become much easier if the division (and the "as-percenting") function matches up dividends and divisors by joining them on tags, so we may want to introduce tag-join-based versions of these functions at some point.
I note that divideSeries, divideSeriesList and asPercent functions are not tag aware. they simply match dividends and divisors in the order they were provided, without regard for which tags they have (at this point i would like to ask you @shanson7 , do you do anything special to make this work for you? as in, how do you assure you have the right pairs being matched up? Do you sort the dividend and divisorList somehow, or do you rely on them being sorted already?)
Additionally, or because of the former, I'm not excited about introducing a new set of functions that replicate the current ones (although possibly merge divideSeries and divideSeriesLists into one), for the sole reason of being able to pass extra parameters to control how labels are set on the outputs. That's just messy.
Thus, I'm hopeful we can simply agree on 1 behavior which we would consider sane and good enough, which doesn't require any flags, nor worries about backwards compatibility because we can just see it as bugfix for the current lack of tags setting.
So, my proposal is simply that we tweak divideSeries, divideSeriesLists, and asPercent, such that each output series gets all tags from the dividend, as well as tags from the divisor that have keys that weren't present in the divisor, as well as an aggregatedBy tag just like aggregations.

this is @DanCech 's proposal, but no config flag
this errs on the side of including more tags rather than fewer. If the user wants to get rid of some tags, we could simply write another function "stripTags" or something which strips a set of tags from series.

How does this sound ?

Gave this some more thought and discussed a bit internally with some set theory nerds at Grafana ( :) )
First of all, I think figuring out which tags to set on the results may become much easier if the division (and the "as-percenting") function matches up dividends and divisors by joining them on tags, so we may want to introduce tag-join-based versions of these functions at some point.
I note that divideSeries, divideSeriesList and asPercent functions are not tag aware. they simply match dividends and divisors in the order they were provided, without regard for which tags they have (at this point i would like to ask you @shanson7 , do you do anything special to make this work for you? as in, how do you assure you have the right pairs being matched up? Do you sort the dividend and divisorList somehow, or do you rely on them being sorted already?)
Additionally, or because of the former, I'm not excited about introducing a new set of functions that replicate the current ones (although possibly merge divideSeries and divideSeriesLists into one), for the sole reason of being able to pass extra parameters to control how labels are set on the outputs. That's just messy.
Thus, I'm hopeful we can simply agree on 1 behavior which we would consider sane and good enough, which doesn't require any flags, nor worries about backwards compatibility because we can just see it as bugfix for the current lack of tags setting.
So, my proposal is simply that we tweak divideSeries, divideSeriesLists, and asPercent, such that each output series gets all tags from the dividend, as well as tags from the divisor that have keys that weren't present in the divisor, as well as an aggregatedBy tag just like aggregations.

this is @DanCech 's proposal, but no config flag
this errs on the side of including more tags rather than fewer. If the user wants to get rid of some tags, we could simply write another function "stripTags" or something which strips a set of tags from series.

How does this sound ?

one more thing.. there is no precedent for somethig like aggregatedBy: divide or agregatedBy: asPercent. That's normal because there are no such aggregation functions. Maybe I took @DanCech too literally when he said "along with a new tag to be consistent with the behavior of other aggregation-type functions".
I do see other functions settings describing their own name, as well as a parameter e.g. series.tags['pow'] = factor or for when no parameter available they just set the function name series.tags['sigmoid'] = 'sigmoid'.
For consistency then, let me retract the aggregatedBy proposal, and instead do something like:
series.tags['divideSeries'] = 'divideSeries'
series.tags['divideSeriesLists'] = 'divideSeriesLists'
series.tags['asPercent'] = 'asPercent'
I don't see it as particularly useful, but many other series do it, so ...

one more thing.. there is no precedent for somethig like aggregatedBy: divide or agregatedBy: asPercent. That's normal because there are no such aggregation functions. Maybe I took @DanCech too literally when he said "along with a new tag to be consistent with the behavior of other aggregation-type functions".
I do see other functions settings describing their own name, as well as a parameter e.g. series.tags['pow'] = factor or for when no parameter available they just set the function name series.tags['sigmoid'] = 'sigmoid'.
For consistency then, let me retract the aggregatedBy proposal, and instead do something like:
series.tags['divideSeries'] = 'divideSeries'
series.tags['divideSeriesLists'] = 'divideSeriesLists'
series.tags['asPercent'] = 'asPercent'
I don't see it as particularly useful, but many other series do it, so ...

I note that divideSeries, divideSeriesList and asPercent functions are not tag aware. they simply match dividends and divisors in the order they were provided, without regard for which tags they have (at this point i would like to ask you @shanson7 , do you do anything special to make this work for you? as in, how do you assure you have the right pairs being matched up? Do you sort the dividend and divisorList somehow, or do you rely on them being sorted already?)

Yes, we use aliasByTags and sortByName on the divisor and dividend series to make sure the order is correct.

So, my proposal is simply that we tweak divideSeries, divideSeriesLists, and asPercent, such that each output series gets all tags from the dividend, as well as tags from the divisor that have keys that weren't present in the divisor

This sounds reasonable. Would it still be based on input order or would there be tag matching?

For consistency then, let me retract the aggregatedBy proposal, and instead do something like:
series.tags['divideSeries'] = 'divideSeries'
series.tags['divideSeriesLists'] = 'divideSeriesLists'
series.tags['asPercent'] = 'asPercent'
I don't see it as particularly useful, but many other series do it, so ...

Ditto on not seeing these additional tags as useful. In fact, we often need to work around these tags in grafana by adding groupByTags with the list of tags that we want to keep.

this errs on the side of including more tags rather than fewer. If the user wants to get rid of some tags, we could simply write another function "stripTags" or something which strips a set of tags from series.

I think that a stripTags function would be useful, however it could result in duplicate series (unlike groupByTags which would merge any series that ended up with the same tagset). A version of groupByTags that worked on a set of tags to drop would be interesting, but not strictly required here.

I note that divideSeries, divideSeriesList and asPercent functions are not tag aware. they simply match dividends and divisors in the order they were provided, without regard for which tags they have (at this point i would like to ask you @shanson7 , do you do anything special to make this work for you? as in, how do you assure you have the right pairs being matched up? Do you sort the dividend and divisorList somehow, or do you rely on them being sorted already?)

Yes, we use aliasByTags and sortByName on the divisor and dividend series to make sure the order is correct.

So, my proposal is simply that we tweak divideSeries, divideSeriesLists, and asPercent, such that each output series gets all tags from the dividend, as well as tags from the divisor that have keys that weren't present in the divisor

This sounds reasonable. Would it still be based on input order or would there be tag matching?

For consistency then, let me retract the aggregatedBy proposal, and instead do something like:
series.tags['divideSeries'] = 'divideSeries'
series.tags['divideSeriesLists'] = 'divideSeriesLists'
series.tags['asPercent'] = 'asPercent'
I don't see it as particularly useful, but many other series do it, so ...

Ditto on not seeing these additional tags as useful. In fact, we often need to work around these tags in grafana by adding groupByTags with the list of tags that we want to keep.

this errs on the side of including more tags rather than fewer. If the user wants to get rid of some tags, we could simply write another function "stripTags" or something which strips a set of tags from series.

I think that a stripTags function would be useful, however it could result in duplicate series (unlike groupByTags which would merge any series that ended up with the same tagset). A version of groupByTags that worked on a set of tags to drop would be interesting, but not strictly required here.

This sounds reasonable. Would it still be based on input order or would there be tag matching?

I propose we don't change how the functions match up dividends and divisors.  Merely that we fix the tags being set on the results.  Tag based joining can eventually be done in a new set of functions (at that point we can also address other things, such as having 1 division function regardless of the number of inputs), but that's another project.

This sounds reasonable. Would it still be based on input order or would there be tag matching?

I propose we don't change how the functions match up dividends and divisors.  Merely that we fix the tags being set on the results.  Tag based joining can eventually be done in a new set of functions (at that point we can also address other things, such as having 1 division function regardless of the number of inputs), but that's another project.

I'm still a little worried about changing this behavior backwards incompatibly, since I'm not sure where we will be impacted. However, creating a new divideByTags or something with straight-forward tag behavior sounds like an appealing solution, since users that are impacted can opt-in to using that (and there is value add)

I'm still a little worried about changing this behavior backwards incompatibly, since I'm not sure where we will be impacted. However, creating a new divideByTags or something with straight-forward tag behavior sounds like an appealing solution, since users that are impacted can opt-in to using that (and there is value add)

I'm also voting for new and shiny function, why not?

I'm also voting for new and shiny function, why not?

proliferation of many functions that all work slightly (or not so slightly) differently is not something that leads to an elegant, intuitive user experience. That said, I see how it is a low cost way to introduce new behaviors without breaking compatibility.  Introducing functions should be done thoughtfully and ideally the functions should "feel similar" to other existing ones.   I'm working on a proposal and will keep you posted.

proliferation of many functions that all work slightly (or not so slightly) differently is not something that leads to an elegant, intuitive user experience. That said, I see how it is a low cost way to introduce new behaviors without breaking compatibility.  Introducing functions should be done thoughtfully and ideally the functions should "feel similar" to other existing ones.   I'm working on a proposal and will keep you posted.

Proposal for a tag based division function
introduction
We desire to write a function that looks something along the lines of:
divideSeriesByTag(dividends seriesList, divisors seriesList, *tags)

It is inspired by divideSeries, divideSeriesList and asPercent, with these notes:

allowed inputs:

1 dividend and 1 divisor
a list of dividends and divisors of same length for pairwise matching. no need for separate functions for these use cases
a list of dividends and divisors of unequal length. See below

since asPercent is at its core a division and scaling by 100, we won't implement dedicated "as percent" functionality, instead a division result can fed to scale()
no support for dividing series by a number.  That could be added later without breaking backwards compatibility.
no support for specifying divisors=None and use "nodes" (or tags) to divvy up dividends into groups to be summed and used as divisors (like asPercent).  That behavior is not obvious and not explicit enough

At the core, matching of dividends to divisors is done via tags (either manually specified or auto-discovered. see below), and also a key aspect of this function is how it sets tags on the resulting series. (in contrast to the above mentioned functions where tags from divisor and dividend don't make it into the output series.
terminology

join tags: a set of tags that is used to generate a join key
join key: the list of values (or list of key=value pairs) corresponding to the join tags after ordering them.  Used to match up dividends and divisors with one another.
join key generation: generating the join key by collecting the values of all join tags
extra tags: tags that a dividend or divisor has, but that is not part of the join tags

explanation
So, dividends and divisors are both a seriesList of 1..N series.
(if either or both has no series, a runtime error is triggered)
each series has a set of tags. So let's imagine a set of dividends and a set of divisors
A basic case would look something like:
dividends
* name1{a=1, b=1}
* name1{a=1, b=2}
* name2{a=1, b=1}

divisors
* name3{a=1, b=1, c=3}
* name4{a=1, b=1, c=4}

How do we match up dividends and divisors? We need a set of join tags. We have 2 choices:

have the user specify a set of tags
auto discover the set of tags, e.g. pick the largest set of tag keys that is shared across all dividends (a, b), across all divisors (a,b,c) or both (a,b)

The most simple way, is to

take all keys shared across all dividends and all divisors,
assert that the list of join keys is not empty.  error otherwise
assert that there are no two dividends or divisors sharing the same join key.  (while this could be resolved by specifying an aggregator callback, this makes things overly complicated. we rather error in this case and tell the user to fix their query, possibly by preparing the input with groupByTags)
possibly, assert that for each divisor join key there is a matching dividend join key.  This feels overly restrictive as it seems quite natural to sometimes have more values on either side of the equation.  In this case we can either discard unmatched series, or generate "MISSING" series, similar to asPercent. I propose the latter, as it is more explicit.

However, there is a benefit to manually specifying the join tags in some special cases. Namely, you can choose to ignore a tag that all series have in common, if it's not relevant for joining (e.g. a "description" tag),
likewise you can choose to include a tag even though not all input series have it, if you want to use it for joining on those series that have it. (see the 3rd item below)
looking at any tag of any given dividend and divisor, there are 6 scenarios. We list them below along with possible behaviors.

join tag with same val: contributes to a match, obviously
join tag with different val: makes the pair not match
join tag unset on one side: // would not happen for auto join tags, but this allows to use extra tags for joining if they're set (by manually specifying them)

strict: makes the pair not match
loose: considered a match

join tag unset on both sides: ignored
extra tags with same val:

drop
keep on the resulting series

extra tags with different val:

keep the one from dividend
keep the one from divisor
create a new value that is both of them joined together
drop the tag

extra tags with val not set one side

keep if it's set on the dividend
keep if it's set on the divisor
keep either one
drop the tag

It's not so hard to envision situations in which we may want a variety of these behaviors.
The proposal is to have a simple default case, but ample room to customize if needed.
divideSeriesByTag(dividends seriesList, divisors seriesList, options string, *tags)

join tags defaults to the set of tags that are seen across all dividends and divisors, but can be overridden.
to accommodate all possible behaviors outlined above, the user can specify a comma separated list of options.
as demonstrated here it's not possible
to have an options string that can be completely skipped, along with a variadic tags argument.
Thus, the user will always have to specify options, but in the base case they would specify defaults.
They can then set an option out of each of the categories below, to tweak the behavior.
The proposed default is equivalent to "join_strict,extra_same_keep,extra_diff_keep_dividend,extra_unset_keep_any"
one of:

join_strict
join_loose

one of:

extra_same_drop
extra_same_keep

one of:

extra_diff_keep_dividend
extra_diff_keep_divisor
extra_diff_join
extra_diff_drop

one of:

extra_unset_keep_dividend
extra_unset_keep_divisor
extra_unset_keep_any
extra_unset_drop

Proposal for a tag based division function
introduction
We desire to write a function that looks something along the lines of:
divideSeriesByTag(dividends seriesList, divisors seriesList, *tags)

It is inspired by divideSeries, divideSeriesList and asPercent, with these notes:

allowed inputs:

1 dividend and 1 divisor
a list of dividends and divisors of same length for pairwise matching. no need for separate functions for these use cases
a list of dividends and divisors of unequal length. See below

since asPercent is at its core a division and scaling by 100, we won't implement dedicated "as percent" functionality, instead a division result can fed to scale()
no support for dividing series by a number.  That could be added later without breaking backwards compatibility.
no support for specifying divisors=None and use "nodes" (or tags) to divvy up dividends into groups to be summed and used as divisors (like asPercent).  That behavior is not obvious and not explicit enough

At the core, matching of dividends to divisors is done via tags (either manually specified or auto-discovered. see below), and also a key aspect of this function is how it sets tags on the resulting series. (in contrast to the above mentioned functions where tags from divisor and dividend don't make it into the output series.
terminology

join tags: a set of tags that is used to generate a join key
join key: the list of values (or list of key=value pairs) corresponding to the join tags after ordering them.  Used to match up dividends and divisors with one another.
join key generation: generating the join key by collecting the values of all join tags
extra tags: tags that a dividend or divisor has, but that is not part of the join tags

explanation
So, dividends and divisors are both a seriesList of 1..N series.
(if either or both has no series, a runtime error is triggered)
each series has a set of tags. So let's imagine a set of dividends and a set of divisors
A basic case would look something like:
dividends
* name1{a=1, b=1}
* name1{a=1, b=2}
* name2{a=1, b=1}

divisors
* name3{a=1, b=1, c=3}
* name4{a=1, b=1, c=4}

How do we match up dividends and divisors? We need a set of join tags. We have 2 choices:

have the user specify a set of tags
auto discover the set of tags, e.g. pick the largest set of tag keys that is shared across all dividends (a, b), across all divisors (a,b,c) or both (a,b)

The most simple way, is to

take all keys shared across all dividends and all divisors,
assert that the list of join keys is not empty.  error otherwise
assert that there are no two dividends or divisors sharing the same join key.  (while this could be resolved by specifying an aggregator callback, this makes things overly complicated. we rather error in this case and tell the user to fix their query, possibly by preparing the input with groupByTags)
possibly, assert that for each divisor join key there is a matching dividend join key.  This feels overly restrictive as it seems quite natural to sometimes have more values on either side of the equation.  In this case we can either discard unmatched series, or generate "MISSING" series, similar to asPercent. I propose the latter, as it is more explicit.

However, there is a benefit to manually specifying the join tags in some special cases. Namely, you can choose to ignore a tag that all series have in common, if it's not relevant for joining (e.g. a "description" tag),
likewise you can choose to include a tag even though not all input series have it, if you want to use it for joining on those series that have it. (see the 3rd item below)
looking at any tag of any given dividend and divisor, there are 6 scenarios. We list them below along with possible behaviors.

join tag with same val: contributes to a match, obviously
join tag with different val: makes the pair not match
join tag unset on one side: // would not happen for auto join tags, but this allows to use extra tags for joining if they're set (by manually specifying them)

strict: makes the pair not match
loose: considered a match

join tag unset on both sides: ignored
extra tags with same val:

drop
keep on the resulting series

extra tags with different val:

keep the one from dividend
keep the one from divisor
create a new value that is both of them joined together
drop the tag

extra tags with val not set one side

keep if it's set on the dividend
keep if it's set on the divisor
keep either one
drop the tag

It's not so hard to envision situations in which we may want a variety of these behaviors.
The proposal is to have a simple default case, but ample room to customize if needed.
divideSeriesByTag(dividends seriesList, divisors seriesList, options string, *tags)

join tags defaults to the set of tags that are seen across all dividends and divisors, but can be overridden.
to accommodate all possible behaviors outlined above, the user can specify a comma separated list of options.
as demonstrated here it's not possible
to have an options string that can be completely skipped, along with a variadic tags argument.
Thus, the user will always have to specify options, but in the base case they would specify defaults.
They can then set an option out of each of the categories below, to tweak the behavior.
The proposed default is equivalent to "join_strict,extra_same_keep,extra_diff_keep_dividend,extra_unset_keep_any"
one of:

join_strict
join_loose

one of:

extra_same_drop
extra_same_keep

one of:

extra_diff_keep_dividend
extra_diff_keep_divisor
extra_diff_join
extra_diff_drop

one of:

extra_unset_keep_dividend
extra_unset_keep_divisor
extra_unset_keep_any
extra_unset_drop

Btw, i wouldn't commit to implement all the options first go. I would implement at least the default behavior and a couple of the options, but the implementation phase might reveal some of the options would make things too complicated.

Btw, i wouldn't commit to implement all the options first go. I would implement at least the default behavior and a couple of the options, but the implementation phase might reveal some of the options would make things too complicated.

assert that the list of join keys is not empty. error otherwise

What if you want to divide a number of input series by a single tag-less series (e.g. the result of sum)? Why shouldn't this be allowed?

assert that there are no two dividends or divisors sharing the same join key.

Why would we need to assert this? What if there are "extra" tags that would make the result unique? Imagine we had some metric like "CPU usage per process"  and we want to divide by the total CPU capacity of the host . Matching on just host should be fine, with a unique series output per processName/host combination.
process.cpu.util;processName=X1;host=Y
process.cpu.util;processName=X2;host=Y

host.cpu.capacity;host=Y
divideByTags(process.cpu.util, host.cpu.capacity);processName=X1;host=Y
divideByTags(process.cpu.util, host.cpu.capacity);processName=X2;host=Y
Theoretically, these 2 would have the same join key (host=Y). I could see how using the extra_unset_drop you would get multiple result series with the same name/tags but while possibly visually confusing in grafana is something you can easily do today with the alias functions.

assert that the list of join keys is not empty. error otherwise

What if you want to divide a number of input series by a single tag-less series (e.g. the result of sum)? Why shouldn't this be allowed?

assert that there are no two dividends or divisors sharing the same join key.

Why would we need to assert this? What if there are "extra" tags that would make the result unique? Imagine we had some metric like "CPU usage per process"  and we want to divide by the total CPU capacity of the host . Matching on just host should be fine, with a unique series output per processName/host combination.
process.cpu.util;processName=X1;host=Y
process.cpu.util;processName=X2;host=Y

host.cpu.capacity;host=Y
divideByTags(process.cpu.util, host.cpu.capacity);processName=X1;host=Y
divideByTags(process.cpu.util, host.cpu.capacity);processName=X2;host=Y
Theoretically, these 2 would have the same join key (host=Y). I could see how using the extra_unset_drop you would get multiple result series with the same name/tags but while possibly visually confusing in grafana is something you can easily do today with the alias functions.

I will be picking up with @Dieterbe's proposal. @shanson7 and I discussed this and I agree with his comments (https://github.com/graphite-project/graphite-web/issues/2639#issuecomment-767427909). So for the path forward, the proposal is what Dieter originally said (https://github.com/graphite-project/graphite-web/issues/2639#issuecomment-767099114) with the following changes:

We will allow an empty list of join keys for the divisor(s).
a.Tagless divisors will create a new output series for every dividend they encounter
b. A sane default limit should be enabled with an option to adjust it manually

We will allow more than two dividends and divisors to share the same join keys
a. This may produce multiple output series with the same name/tags, but more often than not it will produce unique output series due to default sane options (specifically extra_unset_keep_any)

A slight change to the naming of the options. Instead of having options such as join_strict and join_loose we will favor option=value. So those two options become join=strict and join=loose.
a. proposed defaults would then change to join=strict,extra_same=keep,extra_diff=keep_dividend,extra_unset=keep_any

Full options list with changes:

join
a. strict
b. loose

extra_same
a. drop
b. keep

extra_diff
a. keep_dividend
b. keep_divisor
c. join
d. drop

extra_unset
a. keep_dividend
b. keep_divisor
c. keep_any
d. drop

max_outseries (not the best name, open to suggestions).

request from Bloomberg, switching over to Graphite v5.  How big an effort? for someone proficient in python, 2 weeks? creating the function in Graphite-web, and then it would go into Graphite V5.


Is there a plan to support Emoji?

:smile:

be more specific, please?

😄

Have this supported yet? I think its a good piont~

Good. It can be implemented as an extension.

Any progress or solution on Emoji? Is anyone working on such extension?

Any progress?

We have no plans to provide emoji for the time being.

I have written a plugin to insert emoji icons in my blog project:
https://github.com/eshengsky/iBlog/blob/master/static/editor-emoji-plugin.js
Preview here:
https://skysun.name/guestbook
You can refer to it if you need:smile:


Add Runme to recommended extensions options

The recommendation dialog to add Runme to recommended extensions should include the following options:

Dismiss option per workspace
Dismiss option globally

Duplicate of https://github.com/stateful/vscode-runme/issues/615


Missing parts of central JLM

See here - compared to the old map.
In a quick search it seems like gush# 30048, for example, is missing from the CSV. Can you verify? Perhaps we'll need to create a topojson for the entire CSV, where we can easily locate missing parts and show to MAPI.

Yes - some Gushim are missing from the original file.
Similar issue was found in GovMap
Email was sent to MAPI to verify

This issue is not related to the code but to the base files. Need to add a some sort of test to verify that all Gushim have been received.


Typo

Challenge http://www.freecodecamp.com/challenges/waypoint-style-the-html-body-element has an issue. Please describe how to reproduce it, and include links to screenshots if possible.
Well, I just wanted to let you know that there is a grammatical derp on the page. There should be an "a" after "has" in the sentence, "Every html page has 'body' element. Okay, well.. Have an awesome day!

working in this issue


Upgrade ray and commit rcf before catalog commit

This commit fixes two main issues:

Upgrade Ray version to >=2.20.0 which has a fix for the issue https://github.com/ray-project/ray/issues/45311.
The new Ray version will also allow us to enable structured logging for ray logs.
Removes usage of BlockWritePathProvider as it's no longer supported in ray data as of 2.30.0
Fixes an unique issue when committing RCF (scenario explained below)

Before: RCF read failure scenario
Below are runs in chronological order:
** Run 1**

Happy case. Nothing goes wrong.

** Run 2**

Incremental run uses the previous round completion file.
Commits a new partition.
But fails to write RCF to S3 due to some reason (this is more likely when s3 throttles or the cluster is terminated due to some other issue)

** Run 3 **

Tries to read from the stale RCF as we didn't write new RCF in run 2.
Fails to read because the catalog has already deleted the files.

All subsequent runs fail.
After: RCF read failure scenario
Assume Run 1 goes through.
** Run 2 **

Incremental run uses the previous round completion file.
If RCF could not written, partition is never commit so RCF and partition are consistent. Assume that RCF was written successfully.
Commit to catalog fails for some reason (same as explained above).

** Run 3 **

This PR ensures that previous RCF is written based on the current partition locator in the destination. So, correct RCF is read.
Compaction runs successfully even if we saw some intermittent failure.

If RCF could not written, partition is never commit so RCF and partition are consistent. Assume that RCF was written successfully.

The order should be reversed based on the changes right? It's partition failed to commit first, then RCF also failed.

Thanks for the fix!


Fix for anonymus  acces

In case of an anonymus acces function the Invokeurl is something like
https://funcxxxxxxx.azurewebsites.net/api/someFunction
instead of
https://funcxxxxxx.azurewebsites.net/api/someFunc?code=KftLdW3_D83xc3G9ma6eXzUT_7shgpXLYdMCgjopFrZNAzFuLWYeng==
The old code did not support the first syntax.

I admit this is a rather 'simple' fix. But I didn't want to write all kind of very funky regexp :)


refactor(ObserverLocator): remove O.o strategy

The Object.observe proposal has been withdrawn from TC39: https://esdiscuss.org/topic/an-update-on-object-observe

this is ready

@jdanyow Please go ahead and merge.


SAK-40672 - Javascript refactor and some minor UI fixes

@adrianfish move JS logic to the Javascript template and use namespaces to avoid conflict, also small UI fixes and rename some id's to be more clear.
Thanks Adrian!


Add my okasaki-pfds package

Thanks.

Please note that you are encouraged to add a brief description to your .ipkg file. This will then be shown in the STATUS table of the pack collection. You add such a description any time you feel like it, and it will then be included with the next nightly.


wire-up-render-extrusion-and-render-tubes

[!WARNING]
This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

#228  👈
#225 : 2 other dependent PRs (#220 , #227 )
develop

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @sophiedeziel and the rest of your teammates on  Graphite


'About' Participedia Homepage Tagline/Intro

From Carol Tojeiro (Maxwell RA):
Are we assuming however that those who open the website already know what participedia is about? I know there’s an “about” tab that you can find on the top left corner of the page. Do we want to include a 1-2 sentence what participedia is about? (Just a suggestion).
From Scott: this is just my two cents but I agree that we need some kind of one-liner. I'm not sure that the one Jesi, Paolo, and I came up with is as fitting as it once was. I think we need to have something that stresses our goals of 1) collaborative documentation (like Wikipedia), 2) knowledge sharing (similar to 1 - ie. writing for the community/for the sake of informing, not for personal recognition), and 3) knowledge mobilization and/or creation (inspiring research, partnerships/networking, innovation, etc.)

In a similar vein, Mark, Edana & Sule shared the following recommendation a few days ago: "We think the home page needs to clearly state what PP is, what it's for, and what to do with it. As it is, unless you already know what PP is, you wouldn't know, and would just move on. We think the map should be pushed down, giving a bit of space to replicate some of the "welcome" content from the current home page. We think the "Explore, Create, Teach" labels from the current home page (now on the "About" page) work very nicely as suggestions for things to do-- and could be linked to the appropriate places or pages. Unless we get this content up front, we don't think people will get as far as the "About" page."

Is it possible that the feature tour (which is not yet live on xyz but will be soon as per #624) may solve some of this. Once it is live, let's let them have a look and see if that is enough info for new users to understand what to do with the site.

Perhaps simply adding a bit more text to the tour elements that is specific to public participation and participedia's mission related to each site feature could also help. Text for the feature tour is in the copy deck if you want to make some suggestions.

This is also from Mark's daughter: "Need a sentence to introduce Participedia on the home page, but not much more text."
I think this could still be solved by re-writing the tagline and including it somewhere near the top of the page (see my comment at the top^)

Pat also brought my attention to this website which has a similar function to PPedia although a totally different academic field: https://www.zooniverse.org/projects/klmasters/galaxy-zoo-3d
Note how the tagline makes it clear and unambiguous what the purpose of the site is


Build fail on macOS and win10

When building from VS on both win10 and macOS platform I receive build error related to the main ark-net solution. Here is the console output:
Target PostBuildEvent:
    REM Create a NuGet package for this project and place the .nupkg file in the project's output directory.
    REM If you see this in Visual Studio's Error List window, check the Output window's Build tab for the actual error.
    ECHO Creating NuGet package in Post-Build event...
    PowerShell -NoProfile -ExecutionPolicy Bypass -Command "& '/Users/doob/Desktop/UnityBlock/ark-net/ark-net/_CreateNewNuGetPackage/DoNotModify/CreateNuGetPackage.ps1' -ProjectFilePath '/Users/doob/Desktop/UnityBlock/ark-net/ark-net/ark-net.csproj' -OutputDirectory '/Users/doob/Desktop/UnityBlock/ark-net/ark-net/bin/Debug/' -BuildConfiguration 'Debug' -BuildPlatform 'AnyCPU'"
    /var/folders/6r/wkww53sj6gbf1cpmj2lq7ygr0000gn/T/tmp69086b90510749819e89e1ac125a4306.exec.cmd: line 5: unexpected EOF while looking for matching `''
    /var/folders/6r/wkww53sj6gbf1cpmj2lq7ygr0000gn/T/tmp69086b90510749819e89e1ac125a4306.exec.cmd: line 6: syntax error: unexpected end of file
    /Library/Frameworks/Mono.framework/Versions/5.2.0/lib/mono/msbuild/15.0/bin/Microsoft.Common.CurrentVersion.targets(5036,5): error MSB3073: The command "REM Create a NuGet package for this project and place the .nupkg file in the project's output directory.
    /Library/Frameworks/Mono.framework/Versions/5.2.0/lib/mono/msbuild/15.0/bin/Microsoft.Common.CurrentVersion.targets(5036,5): error MSB3073: REM If you see this in Visual Studio's Error List window, check the Output window's Build tab for the actual error.
    /Library/Frameworks/Mono.framework/Versions/5.2.0/lib/mono/msbuild/15.0/bin/Microsoft.Common.CurrentVersion.targets(5036,5): error MSB3073: ECHO Creating NuGet package in Post-Build event...
    /Library/Frameworks/Mono.framework/Versions/5.2.0/lib/mono/msbuild/15.0/bin/Microsoft.Common.CurrentVersion.targets(5036,5): error MSB3073: PowerShell -NoProfile -ExecutionPolicy Bypass -Command "& '/Users/doob/Desktop/UnityBlock/ark-net/ark-net/_CreateNewNuGetPackage/DoNotModify/CreateNuGetPackage.ps1' -ProjectFilePath '/Users/doob/Desktop/UnityBlock/ark-net/ark-net/ark-net.csproj' -OutputDirectory '/Users/doob/Desktop/UnityBlock/ark-net/ark-net/bin/Debug/' -BuildConfiguration 'Debug' -BuildPlatform 'AnyCPU'"" exited with code 1.
Done building target "PostBuildEvent" in project "ark-net.csproj" -- FAILED.

Done building project "ark-net.csproj" -- FAILED.

Build FAILED.

Core/PeerApi.cs(101,21,101,26): warning CS0162: Unreachable code detected
/Library/Frameworks/Mono.framework/Versions/5.2.0/lib/mono/msbuild/15.0/bin/Microsoft.Common.CurrentVersion.targets(5036,5): error MSB3073: The command "REM Create a NuGet package for this project and place the .nupkg file in the project's output directory.
/Library/Frameworks/Mono.framework/Versions/5.2.0/lib/mono/msbuild/15.0/bin/Microsoft.Common.CurrentVersion.targets(5036,5): error MSB3073: REM If you see this in Visual Studio's Error List window, check the Output window's Build tab for the actual error.
/Library/Frameworks/Mono.framework/Versions/5.2.0/lib/mono/msbuild/15.0/bin/Microsoft.Common.CurrentVersion.targets(5036,5): error MSB3073: ECHO Creating NuGet package in Post-Build event...
/Library/Frameworks/Mono.framework/Versions/5.2.0/lib/mono/msbuild/15.0/bin/Microsoft.Common.CurrentVersion.targets(5036,5): error MSB3073: PowerShell -NoProfile -ExecutionPolicy Bypass -Command "& '/Users/doob/Desktop/UnityBlock/ark-net/ark-net/_CreateNewNuGetPackage/DoNotModify/CreateNuGetPackage.ps1' -ProjectFilePath '/Users/doob/Desktop/UnityBlock/ark-net/ark-net/ark-net.csproj' -OutputDirectory '/Users/doob/Desktop/UnityBlock/ark-net/ark-net/bin/Debug/' -BuildConfiguration 'Debug' -BuildPlatform 'AnyCPU'"" exited with code 1.
    1 Warning(s)
    1 Error(s)

Thank you for this // will look into it.
Can you try to install the full .NET framework - at least on windows... from your logs I see that you have errors in MONO framework, maybe the case for error lies there.
In the meanwhile - may i suggest  that you:

checkout from source
install NBITCOIN nuget package
try to do a build...

NuGET is packaged with this in mind... still have to do separate packages..
Let me know how it works /
--chris

Tried all of your suggestions but still receiving same error.

What is the target framework in project settings/ check there if you have mono selected and change to the latest full .Net framework 4.6.2 if I remember correct. I have limited access to computer this week:(

Ok, just did  another test>
1-visual studio 2017
2-opened ark.net.sln file
2-download code and run test
3-all ok
please check your tools and settings/
we have something different...
right click project settings for ark-net
set target framework 4.5.2
best

hm, very strange. I changed target to .Net 4.6.2 and retargeted all the packages that gave warnings to do so but same error. Then I retarget back to .Net 4.5.2 but still same error. I also tried on Win10 machine and had same result. Not sure what I could be doing wrong or why my project is different?

I fixed this by downloading Nuget.exe and placing it in ..\ark-net_CreateNewNuGetPackage\DoNotModify.  The powershell script for building the Nuget package on post build is looking for this.  This will work in windows 10, but will not for mac because it's an exe which is windows only.  If you want to fix this for both platforms you can remove the post build event (delete everything in the box).

According to @sharkdev-j solution, downloading nuget.exe and place into \ark-net_CreateNewNuGetPackage\DoNotModify is working for me. Thank you!

Awesome work guys!!


Add 'ts' support for localeExtension

Describe the solution you'd like
.
└── public
    └── locales
        ├── en
        |   └── common.ts
        └── de
            └── common.ts

This is not possible, next-i18next uses i18next-fs-backend behind the scene.


Landscape View : RenderFlex Overflowed

How Can I show Landscape view of my fancy-on-boarding? Whenever I turn my app in landscape mode, it appears to me that RenderFlex overflowed! So How can I solve it?

@thealiflab does #24 solves your issue ?


Add support (and document it) for private data projects

We need to support private projects, since there are projects (i.e. git repos) that contain specific company data, which can't be publish publicly.
Originally this was done using a recursive search to find a folder that was inside the 'data/other' folder (added to .gitignore), but that was the root cause of the "DoS on Data-Project technique to map projects and project's teams" #108 issue
That issue massive test execution delay, was caused by the recursive search technique, which was transversing the .git folder, which resulted in 88 folders being checked for the maturity-model.json file (instead of the expected 5 folders)
The current solution for #108 removed that recursive search, which means that at the moment only the public projects are being shown (which is a bug).
I'm thinking that a better solution is to split the data folders into two sub folders: public and private (with the private folder path being added to .gitignore)
Once this is in place, a test is needed to confirm that those repos are private and not committed to GitHub

This was done via https://github.com/DinisCruz/Maturity-Models/commit/7eb4c3eded593dc3815415ddd7512f90768525b5#diff-e38202c539840430dbc07d9003c0b346
If you add a folder called private to the data folder, the projects inside it will also be loaded


cls as builtin word for python

(Original issue 880 created by dave_mckean on 2013-06-12T23:35:54.451089+00:00)
Should cls be added to the builtins state in PythonLexer as is self? (As defined in the agile.py lexer)
See PEP8: Function and method arguments

I don't think so; it is probably much more often used as a normal variable than as a "position-0" argument for metaclass methods.


can I used in flutter ?

it show me:
root@well:/home/well/myflutter/customers# flutter run
Running 'flutter packages get' in customers...      12679ms
Error detected in application source code:
error: Unable to read Dart source 'package:sqlite/src/dart-ext:dart_sqlite'.
then ,i have no idea.

also would like to use in flutter

Try SQFlite https://pub.dartlang.org/packages/sqflite


Options for output formatter? instead of seperated gui apps.

https://github.com/charmbracelet/glow
I can use this as markdown formatter for seperated questions like
ollama run phind-codellama 'show me basic python example' | glow
ollama run phind-codellama 'another questions' | glow
I can get rendered output.
But it seems doesn't remember what we said, the context.
I want to set glow default output formatter for answers. like...
ollama set formatter --glow
ollama run phind-codellama
-> get formatted output in conversation by default.

Ollama cli reads stdin as the prompt, delimited by a new line. You can achieve the desired behaviour with something like this:
cat <<EOF | ollama run phind-codellama | glow
show me basic python example
another questions
EOF

This maintains the history since ollama interprets it as a single session. History is not persisted across sessions.

It seems work, but not convenient as I thought.


[docs] selected property is missing for MenuItem and its parents.

Duplicates

[X] I have searched the existing issues

Related page
https://mui.com/material-ui/api/menu-item/
Kind of issue
Missing information
Issue description
I was looking for the type of the selected property for the MenuItem componant and it appears that I could not find any references of this prop in [MenuItem]'s page (https://mui.com/material-ui/api/menu-item/)
or its base component ButtonBase page.
The only reference I could find was in the Menu component, in an exemple.
Maybe a reference to ListItem in the MenuItem page could help people to find those useful props.
Context 🔦
Well, I was reviewing a PR of a junior dev in our company and he write
<MenuItem
  ... stufff
  selected={aStringValue}
/>

He made me doubt, so I just wanted to check if the property should be a "checked" or true value and I wanted to link to the documentation.

Looks like when we move the inheritance from ListItem to ButtonBase, we didn't update the props not not ignore the ones coming from the ListItem component. I am creating a PR to fix this.


[Discuss] Suggest Datasets for Heart disease prediction.

This is a Discussion Issue.
The project needs to implement disease predictions for various diseases.
The diabetes prediction is already implemented.
now, the next one is Heart disease prediction.
for that, a good dataset is required.
suggest your datasets. the final dataset will be selected for further process.

Hello, kindly assign me this isssue.

The dataset should have columns regarding age, sex, chest pain type, resting blood pressure, serum cholesterol,fasting blood sugar, resting electrocardiographic results, maximum heart rate ,exercise induced angina, oldpeak, the slope of the peak exercise ST segment, number of major vessels, & thal. The main objective refers to the presence of heart disease in the patient. It is integer valued from 0 (no presence) to 4. It will help us to find any other trends in heart data to predict certain cardiovascular events or find any clear indications of heart health.
The creators of the  dataset I provided are :
Hungarian Institute of Cardiology. Budapest: Andras Janosi, M.D.
University Hospital, Zurich, Switzerland: William Steinbrunn, M.D.
University Hospital, Basel, Switzerland: Matthias Pfisterer, M.D.
V.A. Medical Center, Long Beach and Cleveland Clinic Foundation: Robert Detrano, M.D., Ph.D.
I have done research regarding collection of best dataset from various sources:
https://www.nature.com/articles/s41597-019-0206-3
https://rstudio-pubs-static.s3.amazonaws.com/396380_639e2f68b09e41a0b05f97b5dc8eb3f2.html
https://archive.ics.uci.edu/ml/datasets/Heart+Disease/
https://towardsdatascience.com/heart-disease-prediction-73468d630cfc
https://www.nature.com/articles/s41598-020-62133-5
https://www.kaggle.com/ronitf/heart-disease-uci
https://www.sciencedirect.com/science/article/pii/S2352914820300125.
Regards
Swarnava

https://www.nature.com/articles/s41598-020-62133-5
https://www.nature.com/articles/s41597-019-0206-3
These papers do not relate to the dataset you provided.

The dataset was published in 1988. it is a very old dataset.
It has only 308 instances.

Considering the above points there are very less chances that we will continue with this dataset.
If you have a better one, suggest it.

Okay


Assigning a variable to a split series doesn't work

I cannot seem to get a variable to contain a list, when I use split in my .es query.  The behavior I am seeing seems like instead of generating a list, the value of the variable is overwritten with what should have been the last value in the list.  I have attached a screenshot of the issue I am seeing

Just noticed that there are two offset='-15m' in the .es query, but the result is the same when it is removed.

@SimoneTosato a good occasion to also check all my fixes in the backport branch haven't mess stuff up

@sabbene as @SimoneTosato found apparently this problem cannot be solved elegantly in Mathlion.
The problem you found is originated by the fact that you can't assign a variable to a split:
.es(split stuff).math(a=source) #does not work
but .es(split stuff).math(source+2) and similar works correctly.
It would be pretty messy to solve: Imagine the case where .es(split stuff a) returns 10 elements and .es(split stuff b) returns 12.
If you assign them to two variables,  math(a+b) would have no sense.
Also the split values are not singularly identifiable, so what timeseries should go with what?
The first with the first, the second with the second would be the easy answer, but it's a very forced and narrow one.
It can be "solved" by smashing all the split series into single variables (using an average taking advantage of ES for example) and then working only on one main split series

A note for @sabbene: TImelion has an (undocumented) variable feature that can help you

Workaround
Simply: do not do assignments. After a split something like med=median(source) will not work, but using just  median(source) will.
A bit annoying.
PR welcome


feat: add new ec2 instance types

default small m6i.large
default medium/large m6i.4xlarge
removed m5.large
add "m5n.large", "m5a.large", "c5a.large", "m6i.large", "m6a.large", "c6i.large", "c6a.large", "m5n.4xlarge", "m5a.4xlarge", "c5a.4xlarge", "m6i.4xlarge", "m6a.4xlarge", "c6i.4xlarge", "c6a.4xlarge"

Are all these instances tested by QA or SRE?


Postman Collection format has changed, gives error in Postman

Hi
I am beginning to work with OData, and was going trough the Learning.
When I was going to download the Postman Collection in Postman at this link:
https://www.odata.org/getting-started/learning-odata-on-postman/
Postman reported that "Collection format V1" is no longer supported, an it have to be converted to format V2.
/Graves

I also stumbled in to this issue, and followed the conversion guidelines specified by postman.
The conversion seem to work fine; the converted collection is here (and can be pasted directly into Postman, just choose "Import ==> Raw text":
{
    "info": {
        "_postman_id": "09ddcd24-002e-e034-8e64-c4a60a80b91a",
        "name": "Postman OData Tutorial",
        "schema": "https://schema.getpostman.com/json/collection/v2.0.0/collection.json"
    },
    "item": [
        {
            "name": "01. Read the service root",
            "request": {
                "method": "GET",
                "header": [],
                "body": {
                    "mode": "formdata",
                    "formdata": []
                },
                "url": "http://services.odata.org/V4/TripPinService/",
                "description": "All REST APIs should have a single entry point from which a generic hypermedia client can navigate to the resources in the service. In the response we see links to three things: 1. The $metadata document that describes the schema of ther service 2. Links to various collections of objects like People and Airports 3. Links to other top-level items like Me (a singleton) and operations."
            },
            "response": []
        },
        {
            "name": "02. Read the service metadata",
            "request": {
                "method": "GET",
                "header": [],
                "body": {
                    "mode": "formdata",
                    "formdata": []
                },
                "url": "http://services.odata.org/V4/TripPinService/$metadata",
                "description": "$metadata is an endpoint in OData services that contains a machine-readable description of the service model including type schemas, available operations, etc."
            },
            "response": []
        },
        {
            "name": "03. Read an entity set",
            "request": {
                "method": "GET",
                "header": [],
                "body": {
                    "mode": "formdata",
                    "formdata": []
                },
                "url": "http://services.odata.org/V4/TripPinService/People",
                "description": "One of the most common responses from a REST API is a collection of resources. In this case we asked for the People collection. For each response, the OData service writes a self-described response (another REST principle) by annotating the response with a context URL. This context URL tells the service that the contents of the response are a collection of things in the People entity set. The @odata.nextLink annotation is present because the server opted to split the result set across multiple pages. The client can also drive paging using $top and $skip, but server-side paging is a mitigation against DoS attacks. The value property contains the bulk of the response. Note that @odata.id and @odata.editLink should generally not be present in the payload unless they deviate from convention. In this case it appears that there is a bug in our sample service. Pretend those properties aren't there."
            },
            "response": []
        },
        {
            "name": "04. Get a single entity from an entity set",
            "request": {
                "method": "GET",
                "header": [],
                "body": {
                    "mode": "formdata",
                    "formdata": []
                },
                "url": "http://services.odata.org/V4/TripPinService/People('russellwhyte')",
                "description": "To get a particular entity from a collection, append a key segment. By default, key segments in OData services are bounded by parens because they may be composite keys, e.g., OrderLine(OrderId=1,LineNumber=1) or alternate keys, e.g., Person(SSN='000-00-0000') and Person(2115) both address the same resource. Some OData services use normal URL segments for key segments, e.g., Orders/1. This is not recommended because of the scenarios mentioned above."
            },
            "response": []
        },
        {
            "name": "05. Get a primitive property",
            "request": {
                "method": "GET",
                "header": [],
                "body": {
                    "mode": "formdata",
                    "formdata": []
                },
                "url": "http://services.odata.org/V4/TripPinService/People('russellwhyte')/FirstName",
                "description": "Even when fetching a primitive property, an object wrapper is returned rather than returning the raw primitive. This is to protect against a JSON vulnerability."
            },
            "response": []
        },
        {
            "name": "06. Get the raw value of a primitive property",
            "request": {
                "method": "GET",
                "header": [],
                "body": {
                    "mode": "formdata",
                    "formdata": []
                },
                "url": "http://services.odata.org/V4/TripPinService/People('russellwhyte')/FirstName/$value",
                "description": "If you really want the raw value of a primitive property, you can get it by appending $value to the URL. Note that the Content-Type header indicates that the content is text/plain."
            },
            "response": []
        },
        {
            "name": "07. Navigate to related entities",
            "request": {
                "method": "GET",
                "header": [],
                "body": {
                    "mode": "formdata",
                    "formdata": []
                },
                "url": "http://services.odata.org/V4/TripPinService/People('russellwhyte')/Friends('scottketchum')/AddressInfo",
                "description": "To navigate the resource graph, keep appending segments representing valid property names as defined in $metadata or in a full metadata response (see query x). In this case we have started from the service root, navigated to the entity set People, navigated to the resource keyed 'russellwhyte', navigated to the Friends property, navigated to the resource keyed 'scottketchum', and finally navigated to the AddressInfo property. Note that the @odata.context URL self-describes the payload."
            },
            "response": []
        },
        {
            "name": "08. Filter a collection",
            "request": {
                "method": "GET",
                "header": [],
                "body": {
                    "mode": "formdata",
                    "formdata": []
                },
                "url": {
                    "raw": "http://services.odata.org/V4/TripPinService/People?$filter=FirstName eq 'Vincent'",
                    "protocol": "http",
                    "host": [
                        "services",
                        "odata",
                        "org"
                    ],
                    "path": [
                        "V4",
                        "TripPinService",
                        "People"
                    ],
                    "query": [
                        {
                            "key": "$filter",
                            "value": "FirstName eq 'Vincent'"
                        }
                    ]
                },
                "description": "The $filter system query option can be used to filter any collection of resources. Note that the response to a filtered collection is a collection of the same type, regardless of the number of matched resources."
            },
            "response": []
        },
        {
            "name": "09. Filter on enumeration properties",
            "request": {
                "method": "GET",
                "header": [],
                "body": {
                    "mode": "formdata",
                    "formdata": []
                },
                "url": {
                    "raw": "http://services.odata.org/V4/TripPinService/People?$filter=Gender eq Microsoft.OData.SampleService.Models.TripPin.PersonGender'Female'",
                    "protocol": "http",
                    "host": [
                        "services",
                        "odata",
                        "org"
                    ],
                    "path": [
                        "V4",
                        "TripPinService",
                        "People"
                    ],
                    "query": [
                        {
                            "key": "$filter",
                            "value": "Gender eq Microsoft.OData.SampleService.Models.TripPin.PersonGender'Female'"
                        }
                    ]
                },
                "description": "You can filter any type of collection in OData services. When referring to a member of enum properties, please don't ignore the namespace for the enum property."
            },
            "response": []
        },
        {
            "name": "10. Filter on nested structures",
            "request": {
                "method": "GET",
                "header": [],
                "body": {
                    "mode": "formdata",
                    "formdata": []
                },
                "url": {
                    "raw": "http://services.odata.org/V4/TripPinService/Airports?$filter=Location/City/Region eq 'California'",
                    "protocol": "http",
                    "host": [
                        "services",
                        "odata",
                        "org"
                    ],
                    "path": [
                        "V4",
                        "TripPinService",
                        "Airports"
                    ],
                    "query": [
                        {
                            "key": "$filter",
                            "value": "Location/City/Region eq 'California'"
                        }
                    ]
                },
                "description": "You can use any related properties in a filter clause by using the same segments used in the path to traverse properties."
            },
            "response": []
        },
        {
            "name": "11. Filter using logic operators",
            "request": {
                "method": "GET",
                "header": [],
                "body": {
                    "mode": "formdata",
                    "formdata": []
                },
                "url": {
                    "raw": "http://services.odata.org/V4/TripPinService/People?$filter=not(contains(FirstName,'Q')) and (AddressInfo/any(ai:ai/City/Region eq 'WA') or AddressInfo/any(ai:ai/City/Region eq 'ID'))",
                    "protocol": "http",
                    "host": [
                        "services",
                        "odata",
                        "org"
                    ],
                    "path": [
                        "V4",
                        "TripPinService",
                        "People"
                    ],
                    "query": [
                        {
                            "key": "$filter",
                            "value": "not(contains(FirstName,'Q')) and (AddressInfo/any(ai:ai/City/Region eq 'WA') or AddressInfo/any(ai:ai/City/Region eq 'ID'))"
                        }
                    ]
                },
                "description": "You can use and, or and not to create more complex filter clauses."
            },
            "response": []
        },
        {
            "name": "12. Filter using any/all operators",
            "request": {
                "method": "GET",
                "header": [],
                "body": {
                    "mode": "formdata",
                    "formdata": []
                },
                "url": {
                    "raw": "http://services.odata.org/V4/TripPinService/People?$filter=Emails/any(e: endswith(e, 'contoso.com'))",
                    "protocol": "http",
                    "host": [
                        "services",
                        "odata",
                        "org"
                    ],
                    "path": [
                        "V4",
                        "TripPinService",
                        "People"
                    ],
                    "query": [
                        {
                            "key": "$filter",
                            "value": "Emails/any(e: endswith(e, 'contoso.com'))"
                        }
                    ]
                },
                "description": "You can use any/all lambda-style filters for collection properties."
            },
            "response": []
        },
        {
            "name": "13. Filter using built-in functions",
            "request": {
                "method": "GET",
                "header": [],
                "body": {
                    "mode": "formdata",
                    "formdata": []
                },
                "url": "http://services.odata.org/V4/TripPinService/People('russellwhyte')/Trips(0)/PlanItems/Microsoft.OData.SampleService.Models.TripPin.Flight",
                "description": "Built-in functions may be leveraged to filter the collection."
            },
            "response": []
        },
        {
            "name": "14. Sort a collection",
            "request": {
                "method": "GET",
                "header": [],
                "body": {
                    "mode": "formdata",
                    "formdata": []
                },
                "url": {
                    "raw": "http://services.odata.org/V4/TripPinService/People?$orderby=length(FirstName) desc,UserName",
                    "protocol": "http",
                    "host": [
                        "services",
                        "odata",
                        "org"
                    ],
                    "path": [
                        "V4",
                        "TripPinService",
                        "People"
                    ],
                    "query": [
                        {
                            "key": "$orderby",
                            "value": "length(FirstName) desc,UserName"
                        }
                    ]
                },
                "description": "You can use the $orderby system query option to specify ordering criteria. You can use many of the functions usable in $filter in $orderby as well."
            },
            "response": []
        },
        {
            "name": "15. Client-side paging",
            "request": {
                "method": "GET",
                "header": [],
                "body": {
                    "mode": "formdata",
                    "formdata": []
                },
                "url": {
                    "raw": "http://services.odata.org/V4/TripPinService/People?$top=2&$skip=2",
                    "protocol": "http",
                    "host": [
                        "services",
                        "odata",
                        "org"
                    ],
                    "path": [
                        "V4",
                        "TripPinService",
                        "People"
                    ],
                    "query": [
                        {
                            "key": "$top",
                            "value": "2"
                        },
                        {
                            "key": "$skip",
                            "value": "2"
                        }
                    ]
                },
                "description": "There are two types of paging in OData services. Servers can enable server-side paging, returning nextLinks that clients can follow to subsequent pages. Clients can drive paging using $top and $skip."
            },
            "response": []
        },
        {
            "name": "16. Counting the elements in a collection",
            "request": {
                "method": "GET",
                "header": [],
                "body": {
                    "mode": "formdata",
                    "formdata": []
                },
                "url": "http://services.odata.org/V4/TripPinService/People/$count",
                "description": "If you want to know how many items meet a condition, you can use the $count path segment. Note that the Content-Type header indicates that the content is text/plain. Although it doesn't work with system query options in the reference service, $count can typically be combined with $filter."
            },
            "response": []
        },
        {
            "name": "17. Expand related entities",
            "request": {
                "method": "GET",
                "header": [],
                "body": {
                    "mode": "formdata",
                    "formdata": []
                },
                "url": {
                    "raw": "http://services.odata.org/V4/TripPinService/People?$expand=Friends,Trips",
                    "protocol": "http",
                    "host": [
                        "services",
                        "odata",
                        "org"
                    ],
                    "path": [
                        "V4",
                        "TripPinService",
                        "People"
                    ],
                    "query": [
                        {
                            "key": "$expand",
                            "value": "Friends,Trips"
                        }
                    ]
                },
                "description": "You can use the $expand system query option to include related resources. The expand clause can include many of the other system query options, enabling left-join type semantics. Also, you can expand <property>/$ref in order to get just the links to the related resources."
            },
            "response": []
        },
        {
            "name": "18. Select the properties",
            "request": {
                "method": "GET",
                "header": [],
                "body": {
                    "mode": "formdata",
                    "formdata": []
                },
                "url": {
                    "raw": "http://services.odata.org/V4/TripPinService/People?$select=FirstName,LastName",
                    "protocol": "http",
                    "host": [
                        "services",
                        "odata",
                        "org"
                    ],
                    "path": [
                        "V4",
                        "TripPinService",
                        "People"
                    ],
                    "query": [
                        {
                            "key": "$select",
                            "value": "FirstName,LastName"
                        }
                    ]
                },
                "description": "You can use the $select system query option to only return specified properties."
            },
            "response": []
        },
        {
            "name": "19. Request full metadata",
            "request": {
                "method": "GET",
                "header": [],
                "body": {
                    "mode": "formdata",
                    "formdata": []
                },
                "url": {
                    "raw": "http://services.odata.org/V4/TripPinService/People?$format=application/json;odata.metadata=full",
                    "protocol": "http",
                    "host": [
                        "services",
                        "odata",
                        "org"
                    ],
                    "path": [
                        "V4",
                        "TripPinService",
                        "People"
                    ],
                    "query": [
                        {
                            "key": "$format",
                            "value": "application/json;odata.metadata=full"
                        }
                    ]
                },
                "description": "By default OData services return an extremely compact JSON format. This happens by stripping out all of the metadata that should be calculable by \"smart\" OData clients. For generic hypermedia clients, you can request additional metadata by using the Accept header or $format system query option to request application/json;odata.metadata=full. In this case, we get a bunch of additional annotations in the payload indicating type information and relationships to related resources."
            },
            "response": []
        },
        {
            "name": "20. Casting types",
            "request": {
                "method": "GET",
                "header": [],
                "body": {
                    "mode": "formdata",
                    "formdata": []
                },
                "url": "http://services.odata.org/V4/TripPinService/People('russellwhyte')/Trips(0)/PlanItems/Microsoft.OData.SampleService.Models.TripPin.Flight",
                "description": "Another way of filtering items is to use a type cast segment. In this case we are looking for all of the flights that are part of Russell's trip. A type cast also allows us to append additional path segments that are properties of the subtype."
            },
            "response": []
        },
        {
            "name": "21. Create an entity",
            "request": {
                "method": "POST",
                "header": [
                    {
                        "key": "Content-Type",
                        "value": "application/json"
                    }
                ],
                "body": {
                    "mode": "raw",
                    "raw": "{\n  \"UserName\": \"miathompson\",\n  \"FirstName\": \"Mia\",\n  \"LastName\": \"Thompson\",\n  \"Gender\": \"Female\"\n}"
                },
                "url": "http://services.odata.org/V4/TripPinService/People",
                "description": "To create a resource, send a POST to a collection."
            },
            "response": []
        },
        {
            "name": "22. Delete an entity",
            "request": {
                "method": "DELETE",
                "header": [
                    {
                        "key": "If-Match",
                        "value": "W/\"08D2931BACB7D7FD\""
                    }
                ],
                "body": {
                    "mode": "formdata",
                    "formdata": []
                },
                "url": "http://services.odata.org/V4/TripPinService/People('miathompson')",
                "description": "To remove a resource, send an HTTP DELETE to the resource URL. Note: since the People entity set has concurrency enabled, you will need to request the resource and set the If-Match header to the appropriate value to run this request."
            },
            "response": []
        },
        {
            "name": "23. Update an entity",
            "request": {
                "method": "PATCH",
                "header": [
                    {
                        "key": "If-Match",
                        "value": "W/\"08D2931BACB7D7FD\""
                    },
                    {
                        "key": "Content-Type",
                        "value": "application/json"
                    }
                ],
                "body": {
                    "mode": "raw",
                    "raw": "{\n  \"Emails\"<EMAIL_ADDRESS><EMAIL_ADDRESS>                },
                "url": "http://services.odata.org/V4/TripPinService/People('miathompson')",
                "description": "To update a resource, send a PATCH request with the properties you wish to modify. You can also use PUT, but the semantics for put require all properties to be either sent on the wire or reverted to their default values. Note: since the People entity set has concurrency enabled, you will need to request the resource and set the If-Match header to the appropriate value to run this request."
            },
            "response": []
        },
        {
            "name": "24. Invoke an unbound function",
            "request": {
                "method": "GET",
                "header": [],
                "body": {
                    "mode": "formdata",
                    "formdata": []
                },
                "url": "http://services.odata.org/V4/TripPinService/GetNearestAirport(lat = 33, lon = -118)",
                "description": "OData support unbound custom functions. The unbound functions can be invoked either with parameters or not and unbound functions can be viewed in the $metadata. Note: OData functions CANNOT have side effect, so only GET verb is allowed."
            },
            "response": []
        },
        {
            "name": "25. Invoke a bound function",
            "request": {
                "method": "GET",
                "header": [],
                "body": {
                    "mode": "formdata",
                    "formdata": []
                },
                "url": "http://services.odata.org/V4/TripPinService/People('russellwhyte')/Microsoft.OData.SampleService.Models.TripPin.GetFavoriteAirline()",
                "description": "OData support bound custom functions. The bound functions are bounded to a resource. Note: OData functions CANNOT have side effect, so only GET verb is allowed."
            },
            "response": []
        },
        {
            "name": "26. Invoke a bound action",
            "request": {
                "method": "POST",
                "header": [
                    {
                        "key": "Content-Type",
                        "value": "application/json"
                    }
                ],
                "body": {
                    "mode": "raw",
                    "raw": "{\n\"userName\": \"scottketchum\",\n\"tripId\": 0\n}"
                },
                "url": "http://services.odata.org/V4/TripPinService/People('russellwhyte')/Microsoft.OData.SampleService.Models.TripPin.ShareTrip",
                "description": "OData support bound custom actions. Actions can have side effects. So the HTTP verb for an OData action can be GET,POST,PUT,DELETE according to the behavior of the action. In the example below, the action implies a POST."
            },
            "response": []
        }
    ]
}

Thanks @Kilsgaard & @alexbjorlig. I will take a look at this.


Fix #117 Updating PostgreSQL version

Updating PostgreSQL versions

Thanks for your contribution, but you will need to update it in the clevercloud-sdk-rust first. https://github.com/CleverCloud/clevercloud-sdk-rust/blob/main/src/v4/addon_provider/postgresql.rs#L43

Updated https://github.com/CleverCloud/clevercloud-sdk-rust/pull/59


Can't install Semantic UI 2.4.2

Use the format: [Component] Component Should Do X
i.e. [Checkbox] onChange Should Fire When Update Triggered via DOM
Steps
Installed npm installl -g gulp
Installed npm install semantic-ui --save
It threw an error
Expected Result
Semantic UI is installed in my project
Actual Result
Throws an error on install. Should I be loading an older version?

Version
2.4.2

I tried the same thing with npm i<EMAIL_ADDRESS>--save but it threw the same exact errors

Try installing the community fork, #7035

Does it have the same updates as Semantic UI?
I want a long term solution I don't want to switch the package mid development.

Semantic hasn't had an update since 2018, fomantic has a ton of bugfixes and extra features. Their last release is less than a week ago. I haven't had any compatibility issues since switching a few years ago.

Awesome.
What about Semantic UI React is it a official plugin?  https://react.semantic-ui.com/

iirc SUIR is official, but they are not fully compatible with FUI yet. Looks like they're working on a compat layer, though I do not know the status of it. I do not use SUIR.


Unsound Use of CStr::from_ptr in add_architecture_hint

Description
The add_architecture_hint function uses unsafe { CStr::from_ptr(hint) } to interpret a raw C string pointer (*mut c_char) as a CStr. This approach assumes that the pointer is valid, properly aligned, non-null, and null-terminated. If any of these conditions are violated, the program will invoke undefined behavior (UB). The function does not validate the hint pointer or its contents, making it unsound.
https://github.com/intel/tsffs/blob/1556d0facc804e35b5696622868ea7d0c3a4b989/src/interfaces/config.rs#L30
pub fn add_architecture_hint(&mut self, cpu: *mut ConfObject, hint: *mut c_char) -> Result<()> {
    let hint = unsafe { CStr::from_ptr(hint) }.to_str()?;
    let processor_number = get_processor_number(cpu)?;
    debug!(
        self.as_conf_object(),
        "add_architecture_hint({processor_number}, {hint})"
    );
    self.architecture_hints
        .insert(processor_number, ArchitectureHint::from_str(hint)?);

    Ok(())
}

Problems:
this function is a pub function, so I assume user can control the hint field, it cause some problems.

Unchecked Pointer Validity:
The function does not verify that hint is a valid pointer. If hint is null, misaligned, or invalid, the call to CStr::from_ptr results in undefined behavior.
Null-Termination Requirement:
CStr::from_ptr requires the string to be null-terminated. If the input pointer does not point to a null-terminated string, the function will read out of bounds, causing undefined behavior.

The function does not document or enforce safety requirements for the hint parameter, leaving it up to the caller to ensure validity. This violates Rust's safety principles and makes the function unsound.
Suggestion

mark this function as unsafe and provide safety doc.
add some check in the function body.

Additional Context:
This issue arises from the unsafe handling of raw pointers and unchecked assumptions about input validity. Rust's unsafe constructs should only be used when their safety guarantees can be upheld, and all potential invalid states must be handled explicitly.

same problem for https://github.com/intel/tsffs/blob/1556d0facc804e35b5696622868ea7d0c3a4b989/src/interfaces/fuzz.rs#L29 and https://github.com/intel/tsffs/blob/1556d0facc804e35b5696622868ea7d0c3a4b989/src/interfaces/fuzz.rs#L239
 pub fn repro(&mut self, testcase_file: *mut c_char) -> Result<()> {
        let simics_path = unsafe { CStr::from_ptr(testcase_file) }.to_str()?;

        let testcase_file = lookup_file(simics_path)?;

        debug!(self.as_conf_object(), "repro({})", testcase_file.display());

        let contents = read(&testcase_file).map_err(|e| {
            anyhow!(
                "Failed to read repro testcase file {}: {}",
                testcase_file.display(),
                e
            )
        })?;

        self.repro_testcase = Some(contents);

        if self.iterations > 0 {
            // We've done an iteration already, so we need to reset and run
            self.restore_initial_snapshot()?;
            self.get_and_write_testcase()?;
            self.post_timeout_event()?;

            run_alone(|| {
                continue_simulation(0)?;
                Ok(())
            })?;
        }

        Ok(())
    }

pub fn solution(&mut self, id: u64, message: *mut c_char) -> Result<()> {
        let message = unsafe { CStr::from_ptr(message) }.to_str()?;

        debug!(self.as_conf_object(), "solution({id:#x}, {message})");

        self.stop_simulation(StopReason::Solution {
            kind: SolutionKind::Manual,
        })?;

        Ok(())
    }

ping?

Hi @lwz23,
Thanks for this bug report !
I can have a look at it in 2 weeks.

Thanks for your reply. I go through those code again and find all those from_ptr result are handled by ?, so, in suppose there will be a panic instead of abort if arg is invalid. in short, I think maybe those functions just need safety document but won't trigger UB. thanks.


Use the Array Index to Query for a Field in an Embedded Document

Hi,
is it possible with LiteDB to use an array index to query for a field in an embedded document?
In mongoDB, it´s possible to query just in all first objects of an array with "field.index."
db.inventory.find( { 'instock.0.qty': { $lte: 20 } } )
Thank you.

Hi @georglang, keys are not indexes with array position. So when you index a array field, all values are indexes as a multiple values only. If you want use indexed values you need use fixed position array with virtual indexes: Like here https://github.com/mbdavid/LiteDB/issues/530#issuecomment-286427325

Hi! With the objective of organizing our issues, we are closing old unsolved issues. Please check the latest version of LiteDB and open a new issue if your problem/question/suggestion still applies. Thanks!


Update custom menu styles

This PR fixes #148587 by updating the following menu styles:

Background color on default dark and base dark theme for better separation
Foreground color on base dark theme for better contrast
100% width separators with updated colors
Slight increase in item height
Border radius on context menu
Removes 1px border from items

To do:

[ ] Fix focus border jank on hc themes (@sbatten I'm attempting to use an existing rule that addressed this only for HC themes, but using :host-context doesn't seem to have any effect here.

Before

After

There is a problem with the corners

You can see the rounded corner, then a sharp corner, then the shadows

Are you intentionally making the keyboard shortcuts use disabled styling when not focused?

High contrast separator margins are not consistent with other themes

Can you clarify the todo above?

This rule prevented the behavior in the gif below. A side effect of that rule was a that it rendered an unwanted thin border around all menu items. I'm trying to scope it to hc themes like this only to prevent the focus outline janking the layout around, but it doesn't seem to work. Could use your input here.

How did we choose the corner radius? On windows 11, I find the native UI a lot more round, which maybe we could match?

It's roughly similar to what we use for our icon buttons. I'll take a closer look at Win 11 for inspiration. Will also look at the menu buttons themselves.

Are you intentionally making the keyboard shortcuts use disabled styling when not focused?

They are using descriptionForeground to be less prominent on purpose. Looks like not every theme maintains great contrast though, so I'll see what I can do. Will fix the hc separator padding as well.

Many thanks to @sbatten for helping our with the high contrast layout shift fix 🙏
I also fixed the following:

Border radius applies to all themes and matches icon button @ 5px
Box shadow respects border-radius
Separators maintain correct width and margin/padding

https://user-images.githubusercontent.com/25163139/168148212-60b6a949-6142-400a-987a-2dd5fb17b8fb.mp4