Notifications

This PR addresses issue #228.
It adds a new model "notifications" which has a number of events that can be set up to trigger notifications to be sent by mail and/or as a Slack message. This can be done globally as well as on a per-user basis, so individual users can set up notifications of their own without needing to request assistance from an admin. Destinations for personal notifications are the user's mail address and Slack username, respectively.
Initial events that have been setup to offer notifications are:

Engagement added
Test added
Results added to test
Report created

Slack messages are sent using "django_slack" which has been added to the INSTALLED_APPS section of settings.dist.py.

@grendel513 do you have time to review?

Hi @patriknordlen awesome stuff as usual :) would you please add a section to https://github.com/OWASP/django-DefectDojo/blob/master/docs/features.rst documenting how to setup the integration?
https://github.com/OWASP/django-DefectDojo/blob/master/docs/features.rst generates
http://defectdojo.readthedocs.io/en/latest/features.html

Does this replace or integrate with the current alerts in dojo?
I think it should replace them, notifications and alerts seem redundant and
alerts needs a revamp anyway.
On Tue, Jul 11, 2017 at 11:47 AM Greg Anderson<EMAIL_ADDRESS>wrote:

@grendel513 https://github.com/grendel513 do you have time to review?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/OWASP/django-DefectDojo/pull/318#issuecomment-314504575,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAOQ3rJqMqzYqwrzR-yvIv4x_cBEjcRpks5sM6cxgaJpZM4OUXts
.

@grendel513 it does not replace alerts, it's a pure addition only. It wouldn't be too hard to add alert as a notification type in addition to the ones already in there...I haven't looked into revamping alerts though and build something that automatically does something the other way around (create notifications for anything that pushes an alert), that could possibly be more work. :)

I'm not too keen on having two separate alert/notification mechanisms in
dojo...we need to work on combining them and making sure both
functionalities remain.  Since your notification mechanism appears to be
more mature/robust, I recommend taking a look at the current alert
functionality and finding a way to combine them.
I think this will confuse users who are not aware they are separate an
unrelated.
my two cents
On Tue, Jul 11, 2017 at 1:19 PM, Patrik Nordlén<EMAIL_ADDRESS>wrote:

@grendel513 https://github.com/grendel513 it does not replace alerts,
it's a pure addition only. It wouldn't be too hard to add alert as a
notification type in addition to the ones already in there...I haven't
looked into revamping alerts though and build something that automatically
does something the other way around (create notifications for anything that
pushes an alert), that could possibly be more work. :)
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/OWASP/django-DefectDojo/pull/318#issuecomment-314529494,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAOQ3uTbB09doKybM2IMhFknSjlTLFizks5sM7y-gaJpZM4OUXts
.

Definitely sounds fair. I don't mind working something out, just want to make sure I go for an approach that seems sensible to others as well.
My first thought would be my suggestion above, that is, have alerts work as another notification action. So users could then set up either mail, slack or alert as notification for events. Things that use alerts directly today would then need to be rewritten to use notifications instead, and the logic for which action to take would then be handled there.

I think that sounds like an appropriate approach.  See
https://github.com/OWASP/django-DefectDojo/issues/220 for additional
comments on alerts.  You can see what type of events/situations trigger an
alert in the code here
https://github.com/OWASP/django-DefectDojo/blob/master/dojo/utils.py#L612
I feel we should incorporate those events as well.
Thanks for your willingness to take this on!! We really appreciate it.
Jay
On Tue, Jul 11, 2017 at 5:24 PM, Patrik Nordlén<EMAIL_ADDRESS>wrote:

Definitely sounds fair. I don't mind working something out, just want to
make sure I go for an approach that seems sensible to others as well.
My first thought would be my suggestion above, that is, have alerts work
as another notification action. So users could then set up either mail,
slack or alert as notification for events. Things that use alerts directly
today would then need to be rewritten to use notifications instead, and the
logic for which action to take would then be handled there.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/OWASP/django-DefectDojo/pull/318#issuecomment-314589558,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAOQ3mjIhuDb1G2aB_v9Z4U97w-RiodTks5sM_YbgaJpZM4OUXts
.

I agree that in the long run they should be one entity, but I personally would like slack integration sooner rather than later ¯_(ツ)_/¯. @grendel513 would you be okay with merging as if functional and then we can create a separate issue?

I also think before we combine them we would need to decide what displays where. Right now alerts serve to notify you when 1. something fails 2. something hasn't been completed. As an admin I wouldn't want notifications every time something is opened / closed etc. I see a lot of granularity that needs to be defined that is outside of the issue @patriknordlen is working to solve.

the key phrase here is "in the long run" - it is my experience that those
things left undone become technical debt...if we are in there now, we
should get it done ;)
On Tue, Jul 11, 2017 at 5:38 PM, Greg Anderson<EMAIL_ADDRESS>wrote:

I agree that in the long run they should be one entity, but I personally
would like slack integration sooner rather than later ¯_(ツ)_/¯.
@grendel513 https://github.com/grendel513 would you be okay with
merging as if functional and then we can create a separate issue?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/OWASP/django-DefectDojo/pull/318#issuecomment-314591978,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAOQ3lTcjR6821rEqCl9F6llsDeqjSJOks5sM_ligaJpZM4OUXts
.

One thing about current alerts is that they're not necessarily triggered by events. Things like stale or upcoming engagements/scans are not "generated" so much as they're returned when specifically asking for that info. To move that over to run via notifications there would have to be something like a cron job that runs regularly and collects the same info instead. Unless there's some nifty Django feature that I don't know about that solves this.

@aaronweaver would you please share your thoughts?

I've reworked notifications and alerts a bit so they're integrated now. As for notifications that would require some kind of period check - I'm thinking celery periodic tasks might be possible to use for that? My knowledge of it is limited to that it came up when I googled for if that kind of thing is possible with celery. Opinions, thoughts?

Yes, celery is the right approach, and it is already a part of dojo for
reports and jira.  That should be the way to create tasks that are executed
at regular intervals -
http://docs.celeryproject.org/en/latest/userguide/periodic-tasks.html
On Wed, Jul 12, 2017 at 11:45 AM, Patrik Nordlén<EMAIL_ADDRESS>wrote:

I've reworked notifications and alerts a bit so they're integrated now. As
for notifications that would require some kind of period check - I'm
thinking celery periodic tasks might be possible to use for that? My
knowledge of it is limited to that it came up when I googled for if that
kind of thing is possible with celery. Opinions, thoughts?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/OWASP/django-DefectDojo/pull/318#issuecomment-314828059,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAOQ3oA5uNXhVKpAo_SvhCxbbsZlFamWks5sNPgVgaJpZM4OUXts
.

Thanks for the input @grendel513, I'll get right on it. :)

take a look at
https://github.com/OWASP/django-DefectDojo/blob/master/dojo/tasks.py to see
how these are implemented.  additional tasks can be added to this file to
take care of the notifications.  thanks again for attempting to get this
done
On Wed, Jul 12, 2017 at 11:45 AM, Patrik Nordlén<EMAIL_ADDRESS>wrote:

I've reworked notifications and alerts a bit so they're integrated now. As
for notifications that would require some kind of period check - I'm
thinking celery periodic tasks might be possible to use for that? My
knowledge of it is limited to that it came up when I googled for if that
kind of thing is possible with celery. Opinions, thoughts?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/OWASP/django-DefectDojo/pull/318#issuecomment-314828059,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAOQ3oA5uNXhVKpAo_SvhCxbbsZlFamWks5sNPgVgaJpZM4OUXts
.

Just a short update on this. I've worked on it a bit and have some local changes that I haven't committed yet because I'm still rewriting things here and there.

I've added alerts as another notification option.
I've changed all parts of the application that notify the user by mail or alert in some way to use the new notification functionality instead.
I've implemented periodic tasks with celery that will check things that were previously in get_alerts and send notifications based on that.

In other words, progress is being made, so no need to merge this in its current state :)

@patriknordlen just wondering how this was coming along?

@grendel513 making progress now - I'm back from a 4w vacation which is why there's been a lack of updates. Functionality is done, working now on testing functionality in various situations and ensuring a smooth migration.

Ok! Summary of the changes:

All types of notifications in the application is now handled via the function 'create_notification' in utils.py. This function will in turn send notifications via mail, Slack and/or alerts if this has been enabled either on a global or personal scope in the Notifications page.
Notifications are based on templates found in the dojo/notifications directory. Templates are matched using the 'event' argument to create_notification. If no template is found, the template 'other.tpl'  will be used.
Alerts are no longer loaded in full when loading defectDojo pages. An alert count endpoint is called every 10 seconds, but full alert data is only loaded when the user clicks on the dropdown.
The alerts page now has functionality to select and remove alerts.
Alerts that were previously dynamically retrieved each time a user requested his/her alerts (upcoming engagements, stale engagements) have been moved to be run as periodic tasks using celery beat instead. These tasks will run every hour and create alerts according to their configuration.

@devGregA @grendel513 please have a look - with a PR of this size I'm sure there are problems that I've missed even after testing. For instance I don't have a jira instance handy to see if everything works properly there.

@aaronweaver would you also have time to test?

@devGregA, @aaronweaver, @patriknordlen - I tested the UI as well as the Alerts and email integrations.  All work great.  I have not tested integration with Slack, Hipchat, or Jira.
I think we need to add documentation for this new feature...especially the need to:
pip install django-slack
pip install django-multiselectfield
./manage.py makemigrations
./manage.py migrate
We also need to make sure we mention that the celery process needs to be running for all of this to work.
Thanks!

@grendel513 I have tested the JIRA changes, but not the notifications + JIRA. My suggestion is to get the documentation in as you noted, but then I would be okay with merging and looking at it as 'beta features' I have some code I want to submit relating to handling false positives, but I want to get this done first so @patriknordlen doesn't have to deal with conflicts. Would you be okay with merging once the docs are in place and treating them as 'beta' features for maybe a week? By then I will have plenty of time to thoroughly test the changes at my work.

Sounds good @devGregA! I'll work on the docs tonight.

@devGregA @grendel513 sorry for the delay - work etc. I've added a new section to the 'Features' page covering notifications in brief detail.
django_slack is now gone as a dependency (it's not used anymore, notifications are now sent directly using the requests library) and django-multiselectfield is added as a dependency in setup.py. Users are fine if they run pip install . which will take care of installing all dependencies for them, so I added that command to the list of default commands to run when upgrading. makemigrations and migrate are already there.

Merged. Will roll out at my employer this week for testing.


Add cycle breaker transform

Add CycleBreaker component and associated CycleBreakerTransform. The following code,
import dash_mantine_components as dmc
from dash.exceptions import PreventUpdate
from enrich import DashProxy, html, Output, Input

app = DashProxy()
app.layout = html.Div([
    dmc.NumberInput(id="celsius", label="Celsius"),
    dmc.NumberInput(id="fahrenheit", label="Fahrenheit"),
])

@app.callback(Output("celsius", "value"), Input("fahrenheit", "value"))
def update_celsius(value):
    if value is None:
        raise PreventUpdate()
    return (float(value) - 32) / 9 * 5

@app.callback(Output("fahrenheit", "value"), Input("celsius", "value"))
def update_fahrenheit(value):
    if value is None:
        raise PreventUpdate()
    return float(value) / 5 * 9 + 32

if __name__ == '__main__':
    app.run_server()

raises a error like,
"Error: Dependency Cycle Found: celsius.value -> fahrenheit.value -> celsius.value"
The new transform makes it possible to remove the error simply by mounting the transform with a specification on where to break the cycle,
import dash_mantine_components as dmc
from dash.exceptions import PreventUpdate
from enrich import DashProxy, html, Output, Input, CycleBreakerTransform

cycle_breaks = [("fahrenheit", "value")]
app = DashProxy(transforms=[CycleBreakerTransform(cycle_breaks=cycle_breaks)])
app.layout = html.Div([
    dmc.NumberInput(id="celsius", label="Celsius"),
    dmc.NumberInput(id="fahrenheit", label="Fahrenheit"),
])

@app.callback(Output("celsius", "value"), Input("fahrenheit", "value"))
def update_celsius(value):
    if value is None:
        raise PreventUpdate()
    return (float(value) - 32) / 9 * 5

@app.callback(Output("fahrenheit", "value"), Input("celsius", "value"))
def update_fahrenheit(value):
    if value is None:
        raise PreventUpdate()
    return float(value) / 5 * 9 + 32

if __name__ == '__main__':
    app.run_server()

It guess a nice feature of this transform would be to add an (optional) delay on the update cycle, i.e. not allow updates in the cycle to be faster than some frequency. Currently, if the example is modified with a bug in the conversion of C to F (or F to C), an infinite amount of callback invocations are dispatches as fast as the client/server can execute them.

It seems that the PR is missing the CycleBreaker.py.

With the current implementation of the CycleBreakerTransform it will not be possible to break cycles when using the Dash object directly from the enrich.py. One would have to always create a DashProxy manually, providing all the required transform objects.
Wouldn't it be better (and more inline with how other transform objects work), to make it a configuration of a callback level?
Something like:
...

app = DashProxy(transforms=[CycleBreakerTransform()])

...

@app.callback(Output("celsius", "value"), Input("fahrenheit", "value"), break_cycles=[("fahrenheit", "value")])
def update_celsius(value):
    ...

...

Or potentially even like this (where all inputs of the function would be broken automatically):
@app.callback(Output("celsius", "value"), Input("fahrenheit", "value"), break_cycles=True)
def update_celsius(value):
    ...

This also means that if there is another callback that depends on ("fahrenheit", "value") but does not form a cycle with it, it will not be unnecessarily altered. Like in this example:
...

app.layout = html.Div([
    dmc.NumberInput(id="celsius", label="Celsius"),
    dmc.NumberInput(id="fahrenheit", label="Fahrenheit"),
    html.Div(id="output")
])

@app.callback(Output("celsius", "value"), Input("fahrenheit", "value"))
def update_celsius(value):
    ...

@app.callback(Output("fahrenheit", "value"), Input("celsius", "value"))
def update_fahrenheit(value):
    ...

@app.callback(Output("output", "children"), Input("celsius", "value"), Input("fahrenheit", "value"))
def update_celsius(celsius, fahrenheit):
    return f"It is {celsius} celsius and {fahrenheit} fahrenheit"

...

An implementation at the callback level was actually also my initial though. However, I wasn't sure how the syntax should look like.

My first idea was to add a callback level keyword argument (similar to you suggestion). However, while it seems like a fine option for simple callbacks as in the example, it was less obvious for me what to do with callback with many inputs/outputs. Adding breakers for all of them seems excessive

My next idea was to add dependency level keyword argument, i.e. something like,

@app.callback(Output("celsius", "value", break_cycle=True), Input("fahrenheit", "value"))
def update_celsius(value):
    ...

which makes it possible to specify what cycles to break (and which ones not to). What I don't like about this syntax is that it's not really clear if the break break_cycle keyword belongs to the output of this callback, or this input of the other callback, i.e.
@app.callback(Output("fahrenheit", "value"), Input("celsius", "value", break_cycle=True))
def update_fahrenheit(value):
    ...

or any other callbacks involving the property ("celsius", "value"). I think your initial syntax suffers from the same kind of issue. Ideally, I would like a syntax where there is only one place (preferable an obvious only) to specify the break. Please let me know, if you have more ideas :)
With regard to ingrain with the Dash object, one option would be to add a keyword argument, i.e. the syntax would then be something like,
app = Dash(cycle_breaks=[("fahrenheit", "value")])

Another option would be to add breaks automatically where cycles are detected. But I think this option wouldn't yield the necessary control to the user (there could be many places to break a cycle, and performance might be impacted based on the where the break is performed, i.e. the user should be able to control it).

@TomaszRewak I just checked in CycleBreaker.react.js if that's what you mean? The .py files are never checked in

Ah, yes, I was indeed thinking about the CycleBreaker.react.js
I see your point and I agree - it's better to have a clear single place where this thing needs to be configured. It must be obvious to the user.
Just for completeness, there is one other alternative, but I don't know how do I feel about it myself. You could add a CycleBreakInput class (similarly to how there is a Trigger class for the TriggerTransform). The thing I like least about this approach is that I cant think of any good name for that thing.

Adding breakers for all of them seems excessive

That's true. Probably the overall cleanest solution would be to have a callback-level break_cycles boolean flag and to generate the breakers only for the inputs that indeed belong to a cycle. But for that we would have to implement a cycle detection logic ourselves.
The thing I'm probably most afraid when it comes to the global configuration (Dash(cycle_breaks=[...])) is that example of update_output I gave, where the breaking happens also for a callback that does not participate in the cycle. That's a change in behavior that users might not expect. Dash has internally this feature where if you chain callbacks, it will only apply the result of the final callback calculation once all callbacks on which it depends finish execution (also including dependencies of the dependencies of that callback). It's well described in "Dash App With Chained Callbacks" section of the https://dash.plotly.com/basic-callbacks
Following the docs, for this code:
from dash import Dash, dcc, html, Input, Output

external_stylesheets = ['https://codepen.io/chriddyp/pen/bWLwgP.css']

app = Dash(__name__, external_stylesheets=external_stylesheets)

all_options = {
    'America': ['New York City', 'San Francisco', 'Cincinnati'],
    'Canada': [u'Montréal', 'Toronto', 'Ottawa']
}
app.layout = html.Div([
    dcc.RadioItems(
        list(all_options.keys()),
        'America',
        id='countries-radio',
    ),

    html.Hr(),

    dcc.RadioItems(id='cities-radio'),

    html.Hr(),

    html.Div(id='display-selected-values')
])

@app.callback(
    Output('cities-radio', 'options'),
    Input('countries-radio', 'value'))
def set_cities_options(selected_country):
    return [{'label': i, 'value': i} for i in all_options[selected_country]]

@app.callback(
    Output('cities-radio', 'value'),
    Input('cities-radio', 'options'))
def set_cities_value(available_options):
    return available_options[0]['value']

@app.callback(
    Output('display-selected-values', 'children'),
    Input('countries-radio', 'value'),
    Input('cities-radio', 'value'))
def set_display_children(selected_country, selected_city):
    return u'{} is a city in {}'.format(
        selected_city, selected_country,
    )

if __name__ == '__main__':
    app.run_server(debug=True)

If you change the value of the countries dcc.RadioItems component, Dash will wait until the value of the cities component is updated before calling the final callback. This prevents your callbacks from being called with inconsistent state like with "America" and "Montréal".

Now let's say that we add another RadioItems to out layout with phone extension numbers (+48, +1 etc.). As the extension is dependent on the country, we will add a cyclical callback to update one selection based on the other and vice versa. To break it we would specify that Dash(cycle_breaks=[("countries-radio", "value")]). The problem is that it would not only (as expected) break the cycle between the countries RadioItems and phone extensions RadioItems, but it will also introduce the breaker in those other callbacks that depend on the ("countries-radio", "value") - potentially leading to a situation where the app would display "America" and "Montréal" - which Dash by default protect against.

Ah, I didn't really think about it that way. I was thinking that the intention from the start was to break "all chains involving a specific prop". I guess in principle you are right, that we only want to break specific links in the callback chain(s), though I am not sure how often it will make a difference in practice.
If this is indeed the case, the specification ("fahrenheit", "value") is not sufficient - the information on what links to break is missing. I guess the only sane way to collect this information would be the dependency-level keyword argument.
I am trying to wrap my head around how an implementation of that concept would look. So, for starters, does it make the most sense to specify the cycle break on Input or Output? Or do we need both?

With the dependency level keyword syntax, my working example now looks like this,
import dash_mantine_components as dmc
from dash.exceptions import PreventUpdate
from enrich import Dash, html, Output, Input

app = Dash()
app.layout = html.Div([
    dmc.NumberInput(id="celsius", label="Celsius"),
    dmc.NumberInput(id="fahrenheit", label="Fahrenheit"),
])

@app.callback(Output("celsius", "value"), Input("fahrenheit", "value", break_cycle=True))
def update_celsius(value):
    if value is None:
        raise PreventUpdate()
    return (float(value) - 32) / 9 * 5

@app.callback(Output("fahrenheit", "value"), Input("celsius", "value"))
def update_fahrenheit(value):
    if value is None:
        raise PreventUpdate()
    return float(value) / 5 * 9 + 32

if __name__ == '__main__':
    app.run_server()

I think I agree that this is actually better than before :). @TomaszRewak Du you like the keyword name, i.e. break_cycle? Or do you have a better idea? Or would you prefer a CycleBreakerInput component? A such component would make the code more verbose (which I guess is good), but it would require another import (which I think is bad).

I think the syntax you suggested (Input("fahrenheit", "value", break_cycle=True)) works fine!


DefaultDevelocityBuildLifecycleService shouldn't force IP compatibility for build

DefaultDevelocityBuildLifecycleService is unconditionally using GradleLifecycle#beforeProject API, which is IP compatible. However in case the build running vintage, it can produce CC-serialization-related errors, what is confusing for the non-CC build.

Additional context: https://github.com/gradle/gradle/issues/30385


cargo add deletes comments in TOML unexpectedly under certain conditions

Problem
Running cargo add <EXISTING_DEP> causes the preceding comments to be deleted in Cargo.toml.
Steps

Create a new project with cargo new
Add the following to Cargo.toml:

[dependencies]
# Duis aliqua ullamco incididunt cupidatat Lorem est consequat nostrud mollit veniam fugiat.
# Magna elit anim officia reprehenderit tempor exercitation in do voluptate cillum sunt deserunt duis est.
chrono = "0.4.35"
# Enim dolor veniam ipsum pariatur ullamco laborum veniam cillum dolore aliqua in officia.
# Proident esse cillum mollit adipisicing culpa ad labore labore.
serde = "1.0.197"
# Tempor sunt duis labore dolore consequat labore ad.
# Ipsum eu do amet excepteur sint nisi officia ullamco occaecat eu commodo amet veniam.

Run cargo add chrono/serde (in practise the likely reason you do this is to specify additional features, but that's not necessary for a MRE)
Observe that the two lines of preceding comments are deleted

Possible Solution(s)
No response
Notes
No response
Version
cargo 1.77.0 (3fe68eabf 2024-02-29)
release: 1.77.0
commit-hash: 3fe68eabf93cbf3772bbcad09a9206c783e2de3f
commit-date: 2024-02-29
host: x86_64-unknown-linux-gnu
libgit2: 1.7.2 (sys:0.18.2 vendored)
libcurl: 8.5.0-DEV (sys:0.4.70+curl-8.5.0 vendored ssl:OpenSSL/1.1.1w)
ssl: OpenSSL 1.1.1w  11 Sep 2023
os: EndeavourOS Rolling Release (rolling) [64-bit]

Can you try this on nightly? i believe this was fixed in #13402

Nope. Unless cargo +nightly isn't actually using nightly which I doubt is the case.
$ cargo +nightly -Vv
cargo 1.79.0-nightly (d438c80c4 2024-03-19)
release: 1.79.0-nightly
commit-hash: d438c80c45c24be676ef5867edc79d0a14910efe
commit-date: 2024-03-19
host: x86_64-unknown-linux-gnu
libgit2: 1.7.2 (sys:0.18.2 vendored)
libcurl: 8.6.0-DEV (sys:0.4.72+curl-8.6.0 vendored ssl:OpenSSL/1.1.1w)
ssl: OpenSSL 1.1.1w  11 Sep 2023
os: EndeavourOS Rolling Release (rolling) [64-bit]
$ cargo +nightly add serde
# Same behaviour


Replace all references to "Full .NET" or "desktop .NET" with ".NET Framework"

E.g. in https://github.com/aspnet/EntityFramework.Docs/blob/master/entity-framework/core/miscellaneous/connection-strings.md#full-net-applications

I only found 4 docs with "full .NET" and none with "desktop .NET".  I only changed text, not project names in sample code or article URLs.


How to run renderMathInElement without the CDN?

I'm building an electron app so I don't want to use the CDN. Is there any way around this?
Secondly, how do I render a math block with multiple lines of code? the documentation only gives an example of rendering one like of code.
I tried rendering this,
\begin{vmatrix}
   a & b \\
   c & d
\end{vmatrix}

but the following does't work.
        katex.render("\begin{vmatrix} a & b \\ c & d\end{vmatrix}", theorem, {
            throwOnError: false,
            displayMode: true
    });

Yes, there's no need to use the CDN. Just include the js, css, and fonts directory in your project, e.g. via NPM. I do it this way in my Meteor project.
Your example just isn't escaping the backslashes correctly.
katex.render("\\begin{vmatrix} a & b \\\\ c & d\\end{vmatrix}", theorem, {
            throwOnError: false,
            displayMode: true
    });

If you want to use multiple actual lines, you can use \n; see documentation for JavaScript strings.

I did install using npm. I also copied the katex.js, katex.css and the fonts folder from the latest build release  to my root project directory.  and used the following example code I found but it doesn't render.

The formula $a^2+b^2=c^2$ will be rendered inline, but $$a^2+b^2=c^2$$ will be rendered as a block element.

The formula \(a^2+b^2=c^2\) will be rendered inline, but \[a^2+b^2=c^2\] will be rendered as a block element.

@OnlineVagrant I don't see anything wrong with that in principle. We'd need a complete example (online or e.g. uploaded in Github or Gist) to debug further.
Here is how JavaScript strings work: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/String
(Apologies if you already know this; it's all I meant.)

I've got this issue as well. The browser console says TypeError: o.a is undefined when using auto-render.**min**.js but when using auto-render.js I get external_katex_default.a is undefined. Probably the same thing? The details of the error with the unmodified version is as follows:
renderMathInText http://localhost:1313/katex/auto-render.js: 251
    renderElem http://localhost:1313/katex/auto-render.js: 273
    renderElem http://localhost:1313/katex/auto-render.js: 285
    renderElem http://localhost:1313/katex/auto-render.js: 287
    renderElem http://localhost:1313/katex/auto-render.js: 285
    renderElem http://localhost:1313/katex/auto-render.js: 287
    renderElem http://localhost:1313/katex/auto-render.js: 285
    renderElem http://localhost:1313/katex/auto-render.js: 287
    renderElem http://localhost:1313/katex/auto-render.js: 285
    renderElem http://localhost:1313/katex/auto-render.js: 287

I don't know enough javascript to understand what the issues are, but they're presumably on those lines.

@edemaine I though you meant that there was a function in the katex module that deals with strings :)
I get the following error.

Uncaught ReferenceError: renderMathInElement is not defined

I'm assuming this is because I didn't include the auto render extention CDN. As I don't want to use the CDN.
I reduced the code to the simplest form possible.Please take a look if needed.
KatexExample.zip

@OnlineVagrant You're missing the
It began working as expected.

You shouldn't have to do const katex = require('katex'); if you're including a script tag manually.  Doing so should create a global variable called katex which you can use.

@kevinbarabash It would have been nice to not have to include the script tags manually. However, I eventually removed that line of code as I couldn't figure out  how to use the auto render function using the katex variable. The documentation only shows how to use the render function.

If you're using a build system like webpack or parcel you should be able to do
import katex from "katex";
import "katex/dist/katex.min.css";

in one of your source files.  With webpack you'll need to set up css-loader for this to work properly.

Documentation issues are followed in #1944.


feat: Add support for UUIDs (UuidValue)

Adds support for UUIDs. They are represented as a UuidValue on the dart side, and as an uuid in the database.
Fix #635
This is an alternative implementation to #640.
Close #640
Pre-launch Checklist

[x] I read the Contribute page and followed the process outlined there for submitting PRs.
[x] This update contains only one single feature or bug fix and nothing else. (If you are submitting multiple fixes, please make multiple PRs.)
[x] I read and followed the Dart Style Guide and formatted the code with dart format.
[x] I listed at least one issue that this PR fixes in the description above.
[x] I updated/added relevant documentation (doc comments with ///), and made sure that the documentation follows the same style as other Serverpod documentation. I checked spelling and grammar.
[x] I added new tests to check the change I am making.
[ ] All existing and new tests are passing.
[x] Any breaking changes are documented below.

If you need help, consider asking for advice on the discussion board.
Breaking changes
none

This PR now includes 929e5ac. I'm not sure, why the change occurred while regenerating...


Removed optional behavior (shopt) from scripts

Removed the need for the following requested features in the scripts dev_tools/scripts/update_ginkgo_header.sh and dev_tools/scripts/add_license.sh:

extglob (**) by using find instead
globstar with appropriate arguments for find

Additionally, made sure that commands that are used (like find, sort, ...) are present and do not return an error.
I am not 100% sure if this check is necessary (or if I should also include cat, rm, mv and so on), but I don't think it hurts.
A big difference for add_license.sh is that the working directory is changed to the root ginkgo directory, and the temporary files will be created in dev_tools/scripts instead of inside the build directory. However, we should not have any name-clashes (and the temporary file names are now variables which can be easily altered).
Additionally, now all folders named build and third_party are ignored by add_license.sh, not just these folders inside the top level of the ginkgo folder (I changed that when I saw that it previously affected files in test_install/build on my system).
Performance wise, I ran time ./dev_tools/scripts/add_license.sh with the current status:
real	0m2,927s
user	0m2,957s
sys	0m1,046s

Before the changes (current develop status), it is:
real	0m3,007s
user	0m2,990s
sys	0m1,060s

So, we don't lose performance here.
Results for time ./dev_tools/scripts/update_ginkgo_header.sh with the current status:
real	0m0,055s
user	0m0,048s
sys	0m0,009s

Before:
real	0m0,055s
user	0m0,041s
sys	0m0,017s

So we don't lose performance here either.
TODO:

[ ] Confirm that it works on Mac

Closes #272.

Seems not to work for me:
Scanning dependencies of target generate_ginkgo_header Scanning dependencies of target ginkgo_reference_device Scanning dependencies of target ginkgo_cuda_device Scanning dependencies of target ginkgo_omp_device [  0%] Building CXX object core/devices/reference/CMakeFiles/ginkgo_reference_device.dir/dummy.cpp.o [  1%] Building CXX object core/devices/cuda/CMakeFiles/ginkgo_cuda_device.dir/executor.cpp.o [  1%] Building CXX object core/devices/omp/CMakeFiles/ginkgo_omp_device.dir/executor.cpp.o find: -fprint: unknown primary or operator Exiting due to an error being returned byfind! rm: global_includes.hpp.tmp: No such file or directory make[2]: *** [CMakeFiles/generate_ginkgo_header] Error 1 make[1]: *** [CMakeFiles/generate_ginkgo_header.dir/all] Error 2 make[1]: *** Waiting for unfinished jobs.... [  1%] Built target ginkgo_reference_device [  1%] Built target ginkgo_cuda_device [  1%] Built target ginkgo_omp_device make: *** [all] Error 2

@hartwiganzt Apparently, the -fprint option I was using is not supported on macs, which I did not know. I replaced this parameter with normal redirecting to a file, can you try it again?

Works now! Thanks!

Apparently, on Julius' Mac, the add_license.sh does not work (we manually changed the license in one of the files to check if it works) and get the following error multiple times:
grep: repetition-operator operand invalid

The update_ginkgo_header.sh seems to work fine on his system.

Now it works on Julius's Mac and on my system.
It would be nice if the reviewers could also test it on their systems to confirm that it works properly by just executing dev_tools/scripts/add_license.sh and dev_tools/scripts/update_ginkgo_header.sh. If there is no error message, it should work fine.

works.


KAFKA-5818: KafkaStreams state transitions not correct

PR for 0.11.0 \cc @dguy @guozhangwang @bbejeck

JKD9 does only work on trunk -- so failing test is unrelated. Updated this.

LGTM. Merged to 0.11.0.

Thanks @mjsax & @guozhangwang


Add timeouts to destination connection details

What would you like to be added:
Destination service should include connect, read and write timeouts to be used by the proxy in addition to specifying a port (#48) and service name. This should be per service configuration as services of different types can requires different timeouts based on the workloads.
Why is this needed:
All proxies and LB implementation have timeouts that can be defined for establishing a TCP connection, read timeouts, write timeouts, sometimes referred to as idle timeouts.
/kind feature

/remove-lifecycle stale

xref https://github.com/kubernetes-sigs/service-apis/pull/184

/reopen
/remove-lifecycle rotten

/lifecycle frozen

I've spent some time digging into just how portable this could be, and unfortunately it looks like there is quite a bit of variation on how different projects handle this. One of the most broadly supported concepts is that of a request or read timeout, here are some examples of how that is defined:

GCP: This is the amount of time that the load balancer waits for a backend to return a full response to a request.
Envoy: The amount of time that Envoy will wait for the entire request to be received. The timer is activated when the request is initiated, and is disarmed when the last byte of the request is sent upstream (i.e. all decoding filters have processed the request), OR when the response is initiated.
Traefik: The maximum duration for reading the entire request, including the body.
HAProxy: The maximum accepted time to receive a complete HTTP request without affecting the client timeout… By default, this timeout only applies to the header part of the request, and not to any data. As soon as the empty line is received, this timeout is not used anymore. When combined with “option http-buffer-request”, this timeout also applies to the body of the request.
Skipper: Skipper responds with 504 Gateway Timeout status if obtaining a connection, sending the request, and reading the backend response headers and body takes longer than the configured timeout

At first glance, there appears to be some level of potential portability here, but even these relatively similar definitions have some divergence in what they're doing.
When we start to look a bit more broadly, we can see that implementations like nginx and caddyserver have very different timeout configurations that would simply not be portable with any of what's specified above.
I think @hbagdi summarized this well in a related Slack thread today:

Most implementations define timeouts in very different ways.
Timeouts also change must less often. Sometimes never.
Users want a way to configure timeouts. If they are not portable, that's fine. But they would like to keep timeouts separate from the portable configuration. Meaning, I can reuse most of my xRoute resources and I've to tweak only a couple of resources (BackendPolicy, ServicePolicy or such) when I'm migrating from one to another (or using 2 at the same time).

What we are trying to do here is create a higher-order abstraction in the API to achieve portability. But as we have just realized, portability is not possible at all. So this won't work.

And I think we don't even need that.

What Gateway API aims to provide is a consistent "experience". That doesn't require taking the same set of configuration and moving it from one implementation to another. That is a pipe dream (IMHO).

What would a consistent experience look like? A way to configure implementation-specific timeouts in a consistent manner - using the same location/resource in the API.

I think that should be enough because that gives everyone a good trade-off between maintainability, portability and consistency.

I'm interested in some additional perspective here - did we miss any paths to portability here? If not, maybe we should close out this issue?
/cc @bowei @danehans @hbagdi @jpeach

I think the way @hbagdi laid it out is both clear and correct. There's just too much variability here to really be able to make these settings portable.
But the other part of what he said is probably addressable under this issue, namely having a single spot in the API where you put timeouts, and a standard shape of thing to put there, probably a CRD per underlying implementation or something? That would let an implementation easily check if it supported a particular timeout set, or could translate the timeouts, and if not, keeps the config confined to one spot.

One question:
How many different types of timeouts are supported for each of the variants?
For example, is it suffice to say:
ReadTimeout
WriteTimeout

But the specific (which byte) qualifies is up to the implementation?
Do we need to be 100% proscriptive or is it still useful at a high level and point the user to the specific documentation when they have to get into the weeds?

I think @robscott's review showed that even the definitions of what a particular timeout is are really varied between implementations. So I don't know if even that level of generic timeout is possible (because different implementations will have timeouts that may add up to that timeout or something).

Timeouts in general are a incompatible, but there are some general use cases that might be able to help us

is this is streaming endpoint?
is this a request/response endpoint?
is this a large content upload/download endpoint?

Orthogonal to these issues is how to handle idle sessions.
So maybe an approach for the portable API is to represent timeouts as profiles that an implementation can use to do the right thing (or expand into something specific)?

#713 covers this in some sense.
Safe to close?
/cc @robscott @jpeach @youngnick

Yeah I think that's probably the most fitting solution here.
/close


Determine proper count for orgs

Overview
Define proper count for orgs
Action Items

[ ] Review example image below and determine what the proper x/y numbers below should yield

Resources/Instructions

I am auditing the site and their relevant websites to make sure that the correct information is being displayed. This should be completed by the end of this week (4/1)


output mean+quantiles table + transmission spectrum

results table for each parameter: mean, 16th, 50th, 84th percentiles, +1 and -1sigma (calculated traditionally as 84th-50th and 16th-50th)
transmission spectrum table
tested for emcee, dynesty, lsq, for white and wavelength-dependent light curves + shared fits

Can you please go through your edits and make sure that they are all flake8 compliant? If you're using an editor like VS Code, you can make flake8 check your files continuously while you edit them. Otherwise, you can use flake8 in the command line after pip installing it. Sticking to a standard like flake8 will make sure that all of our code is nice and uniformly formatted. And making sure that all our PRs are all consistent with flake8 is important otherwise someone has to go through and edit every file at some point to make sure they're all compliant (I did this a week ago and it wasn't fun).
I ran the following command in the main "Eureka" folder and got many warnings/errors that are introduced with your edits:
flake8 --exclude="./build/**/*,./docs/**/*,.vscode/*,**/site-packages/**/*,*.c,*.pyx,**/*niriss*.py" --per-file-ignores="**/__init__.py:F401,**/manageevent.py:F841,./demos/**/run_*.py:E402,**/test_*.py:E402,*.py:E722" .
The following are the errors/warnings that were introduced with your edits:
./eureka/S6_planet_spectra/s6_spectra.py:352:80: E501 line too long (99 > 79 characters)
./eureka/S6_planet_spectra/s6_spectra.py:356:13: E303 too many blank lines (2)
./eureka/S6_planet_spectra/s6_spectra.py:381:63: W291 trailing whitespace
./eureka/S6_planet_spectra/s6_spectra.py:382:80: E501 line too long (99 > 79 characters)
./eureka/S6_planet_spectra/s6_spectra.py:384:23: E225 missing whitespace around operator
./eureka/S6_planet_spectra/s6_spectra.py:388:25: E225 missing whitespace around operator
./eureka/S6_planet_spectra/s6_spectra.py:389:80: E501 line too long (116 > 79 characters)
./eureka/S6_planet_spectra/s6_spectra.py:390:37: E128 continuation line under-indented for visual indent
./eureka/S6_planet_spectra/s6_spectra.py:390:38: E225 missing whitespace around operator
./eureka/S6_planet_spectra/s6_spectra.py:395:5: E303 too many blank lines (2)
./eureka/S5_lightcurve_fitting/fitters.py:115:1: W293 blank line contains whitespace
./eureka/S5_lightcurve_fitting/fitters.py:117:53: W291 trailing whitespace
./eureka/S5_lightcurve_fitting/fitters.py:118:57: E202 whitespace before ')'
./eureka/S5_lightcurve_fitting/fitters.py:118:59: W291 trailing whitespace
./eureka/S5_lightcurve_fitting/fitters.py:138:80: E501 line too long (90 > 79 characters)
./eureka/S5_lightcurve_fitting/fitters.py:278:80: E501 line too long (87 > 79 characters)
./eureka/S5_lightcurve_fitting/fitters.py:362:22: E261 at least two spaces before inline comment
./eureka/S5_lightcurve_fitting/fitters.py:366:63: E231 missing whitespace after ','
./eureka/S5_lightcurve_fitting/fitters.py:366:73: E231 missing whitespace after ','
./eureka/S5_lightcurve_fitting/fitters.py:366:78: E231 missing whitespace after ','
./eureka/S5_lightcurve_fitting/fitters.py:366:80: E501 line too long (96 > 79 characters)
./eureka/S5_lightcurve_fitting/fitters.py:366:97: W291 trailing whitespace
./eureka/S5_lightcurve_fitting/fitters.py:367:80: E501 line too long (102 > 79 characters)
./eureka/S5_lightcurve_fitting/fitters.py:371:1: W293 blank line contains whitespace
./eureka/S5_lightcurve_fitting/fitters.py:390:80: E501 line too long (90 > 79 characters)
./eureka/S5_lightcurve_fitting/fitters.py:395:5: E303 too many blank lines (3)
./eureka/S5_lightcurve_fitting/fitters.py:763:38: W291 trailing whitespace
./eureka/S5_lightcurve_fitting/fitters.py:765:80: E501 line too long (87 > 79 characters)
./eureka/S5_lightcurve_fitting/fitters.py:830:5: E303 too many blank lines (2)
./eureka/S5_lightcurve_fitting/fitters.py:832:22: E261 at least two spaces before inline comment
./eureka/S5_lightcurve_fitting/fitters.py:834:1: W293 blank line contains whitespace
./eureka/S5_lightcurve_fitting/fitters.py:836:63: E231 missing whitespace after ','
./eureka/S5_lightcurve_fitting/fitters.py:836:73: E231 missing whitespace after ','
./eureka/S5_lightcurve_fitting/fitters.py:836:80: E501 line too long (97 > 79 characters)
./eureka/S5_lightcurve_fitting/fitters.py:836:98: W291 trailing whitespace
./eureka/S5_lightcurve_fitting/fitters.py:837:80: E501 line too long (102 > 79 characters)
./eureka/S5_lightcurve_fitting/fitters.py:838:1: W293 blank line contains whitespace
./eureka/S5_lightcurve_fitting/fitters.py:858:1: W293 blank line contains whitespace
./eureka/S5_lightcurve_fitting/fitters.py:860:80: E501 line too long (94 > 79 characters)
./eureka/S5_lightcurve_fitting/fitters.py:864:5: E303 too many blank lines (2)
./eureka/S5_lightcurve_fitting/fitters.py:869:5: E303 too many blank lines (2)
./eureka/S5_lightcurve_fitting/fitters.py:954:38: W291 trailing whitespace
./eureka/S5_lightcurve_fitting/fitters.py:989:53: W291 trailing whitespace
./eureka/S5_lightcurve_fitting/fitters.py:990:57: E202 whitespace before ')'
./eureka/S5_lightcurve_fitting/fitters.py:990:59: W291 trailing whitespace
./eureka/S5_lightcurve_fitting/fitters.py:991:1: W293 blank line contains whitespace
./eureka/S5_lightcurve_fitting/fitters.py:993:5: E303 too many blank lines (2)
./eureka/S5_lightcurve_fitting/fitters.py:1011:80: E501 line too long (84 > 79 characters)
./eureka/S5_lightcurve_fitting/fitters.py:1015:5: E303 too many blank lines (2)
./eureka/S5_lightcurve_fitting/fitters.py:1226:80: E501 line too long (93 > 79 characters)
./eureka/S5_lightcurve_fitting/fitters.py:1247:1: W293 blank line contains whitespace
./eureka/S5_lightcurve_fitting/fitters.py:1252:80: E501 line too long (87 > 79 characters)
./eureka/S5_lightcurve_fitting/fitters.py:1258:80: E501 line too long (89 > 79 characters)
./eureka/S5_lightcurve_fitting/fitters.py:1259:80: E501 line too long (83 > 79 characters)
./eureka/S5_lightcurve_fitting/fitters.py:1262:5: E303 too many blank lines (2)
./eureka/S5_lightcurve_fitting/fitters.py:1267:46: F821 undefined name 'ch_number'
./eureka/S5_lightcurve_fitting/fitters.py:1276:29: F821 undefined name 'ch_number'

The following are errors that already existend and you don't need to worry about:
./eureka/S3_data_reduction/background.py:338:41: F821 undefined name 'mask2'
./tests/test_MIRI.py:12:5: F401 'eureka.S1_detector_processing.s1_process as s1' imported but unused
./demos/JWST/run_eureka.py:5:1: F401 'eureka.S1_detector_processing.s1_process as s1' imported but unused

It is also important that you check that your PRs pass all of the tests already written and that you write new tests if you've added a significantly new feature that isn't covered by previous tests (new tests aren't needed for this PR). You can check if your PR passes the current tests by running pytest tests in the main "Eureka" folder (you may need to pip install pytest if you haven't already installed it). Pytest also picks up on one of the same errors flake8 found which is that there is now an undefined variable ch_number in the function save_fit inside the fitters.py file.
Finally, also note that in the docstrings for functions, there needs to be a blank line after "History:" otherwise the docs won't be properly formatted (annoying, I know).

Can you please go through your edits and make sure that they are all flake8 compliant? If you're using an editor like VS Code, you can make flake8 check your files continuously while you edit them. Otherwise, you can use flake8 in the command line after pip installing it. Sticking to a standard like flake8 will make sure that all of our code is nice and uniformly formatted. And making sure that all our PRs are all consistent with flake8 is important otherwise someone has to go through and edit every file at some point to make sure they're all compliant (I did this a week ago and it wasn't fun).
I ran the following command in the main "Eureka" folder and got many warnings/errors that are introduced with your edits: flake8 --exclude="./build/**/*,./docs/**/*,.vscode/*,**/site-packages/**/*,*.c,*.pyx,**/*niriss*.py" --per-file-ignores="**/__init__.py:F401,**/manageevent.py:F841,./demos/**/run_*.py:E402,**/test_*.py:E402,*.py:E722" .
The following are the errors/warnings that were introduced with your edits:
./eureka/S6_planet_spectra/s6_spectra.py:352:80: E501 line too long (99 > 79 characters)
./eureka/S6_planet_spectra/s6_spectra.py:356:13: E303 too many blank lines (2)
./eureka/S6_planet_spectra/s6_spectra.py:381:63: W291 trailing whitespace
./eureka/S6_planet_spectra/s6_spectra.py:382:80: E501 line too long (99 > 79 characters)
./eureka/S6_planet_spectra/s6_spectra.py:384:23: E225 missing whitespace around operator
./eureka/S6_planet_spectra/s6_spectra.py:388:25: E225 missing whitespace around operator
./eureka/S6_planet_spectra/s6_spectra.py:389:80: E501 line too long (116 > 79 characters)
./eureka/S6_planet_spectra/s6_spectra.py:390:37: E128 continuation line under-indented for visual indent
./eureka/S6_planet_spectra/s6_spectra.py:390:38: E225 missing whitespace around operator
./eureka/S6_planet_spectra/s6_spectra.py:395:5: E303 too many blank lines (2)
./eureka/S5_lightcurve_fitting/fitters.py:115:1: W293 blank line contains whitespace
./eureka/S5_lightcurve_fitting/fitters.py:117:53: W291 trailing whitespace
./eureka/S5_lightcurve_fitting/fitters.py:118:57: E202 whitespace before ')'
./eureka/S5_lightcurve_fitting/fitters.py:118:59: W291 trailing whitespace
./eureka/S5_lightcurve_fitting/fitters.py:138:80: E501 line too long (90 > 79 characters)
./eureka/S5_lightcurve_fitting/fitters.py:278:80: E501 line too long (87 > 79 characters)
./eureka/S5_lightcurve_fitting/fitters.py:362:22: E261 at least two spaces before inline comment
./eureka/S5_lightcurve_fitting/fitters.py:366:63: E231 missing whitespace after ','
./eureka/S5_lightcurve_fitting/fitters.py:366:73: E231 missing whitespace after ','
./eureka/S5_lightcurve_fitting/fitters.py:366:78: E231 missing whitespace after ','
./eureka/S5_lightcurve_fitting/fitters.py:366:80: E501 line too long (96 > 79 characters)
./eureka/S5_lightcurve_fitting/fitters.py:366:97: W291 trailing whitespace
./eureka/S5_lightcurve_fitting/fitters.py:367:80: E501 line too long (102 > 79 characters)
./eureka/S5_lightcurve_fitting/fitters.py:371:1: W293 blank line contains whitespace
./eureka/S5_lightcurve_fitting/fitters.py:390:80: E501 line too long (90 > 79 characters)
./eureka/S5_lightcurve_fitting/fitters.py:395:5: E303 too many blank lines (3)
./eureka/S5_lightcurve_fitting/fitters.py:763:38: W291 trailing whitespace
./eureka/S5_lightcurve_fitting/fitters.py:765:80: E501 line too long (87 > 79 characters)
./eureka/S5_lightcurve_fitting/fitters.py:830:5: E303 too many blank lines (2)
./eureka/S5_lightcurve_fitting/fitters.py:832:22: E261 at least two spaces before inline comment
./eureka/S5_lightcurve_fitting/fitters.py:834:1: W293 blank line contains whitespace
./eureka/S5_lightcurve_fitting/fitters.py:836:63: E231 missing whitespace after ','
./eureka/S5_lightcurve_fitting/fitters.py:836:73: E231 missing whitespace after ','
./eureka/S5_lightcurve_fitting/fitters.py:836:80: E501 line too long (97 > 79 characters)
./eureka/S5_lightcurve_fitting/fitters.py:836:98: W291 trailing whitespace
./eureka/S5_lightcurve_fitting/fitters.py:837:80: E501 line too long (102 > 79 characters)
./eureka/S5_lightcurve_fitting/fitters.py:838:1: W293 blank line contains whitespace
./eureka/S5_lightcurve_fitting/fitters.py:858:1: W293 blank line contains whitespace
./eureka/S5_lightcurve_fitting/fitters.py:860:80: E501 line too long (94 > 79 characters)
./eureka/S5_lightcurve_fitting/fitters.py:864:5: E303 too many blank lines (2)
./eureka/S5_lightcurve_fitting/fitters.py:869:5: E303 too many blank lines (2)
./eureka/S5_lightcurve_fitting/fitters.py:954:38: W291 trailing whitespace
./eureka/S5_lightcurve_fitting/fitters.py:989:53: W291 trailing whitespace
./eureka/S5_lightcurve_fitting/fitters.py:990:57: E202 whitespace before ')'
./eureka/S5_lightcurve_fitting/fitters.py:990:59: W291 trailing whitespace
./eureka/S5_lightcurve_fitting/fitters.py:991:1: W293 blank line contains whitespace
./eureka/S5_lightcurve_fitting/fitters.py:993:5: E303 too many blank lines (2)
./eureka/S5_lightcurve_fitting/fitters.py:1011:80: E501 line too long (84 > 79 characters)
./eureka/S5_lightcurve_fitting/fitters.py:1015:5: E303 too many blank lines (2)
./eureka/S5_lightcurve_fitting/fitters.py:1226:80: E501 line too long (93 > 79 characters)
./eureka/S5_lightcurve_fitting/fitters.py:1247:1: W293 blank line contains whitespace
./eureka/S5_lightcurve_fitting/fitters.py:1252:80: E501 line too long (87 > 79 characters)
./eureka/S5_lightcurve_fitting/fitters.py:1258:80: E501 line too long (89 > 79 characters)
./eureka/S5_lightcurve_fitting/fitters.py:1259:80: E501 line too long (83 > 79 characters)
./eureka/S5_lightcurve_fitting/fitters.py:1262:5: E303 too many blank lines (2)
./eureka/S5_lightcurve_fitting/fitters.py:1267:46: F821 undefined name 'ch_number'
./eureka/S5_lightcurve_fitting/fitters.py:1276:29: F821 undefined name 'ch_number'

The following are errors that already existed and you don't need to worry about:
./eureka/S3_data_reduction/background.py:338:41: F821 undefined name 'mask2'
./tests/test_MIRI.py:12:5: F401 'eureka.S1_detector_processing.s1_process as s1' imported but unused
./demos/JWST/run_eureka.py:5:1: F401 'eureka.S1_detector_processing.s1_process as s1' imported but unused

It is also important that you check that your PRs pass all of the tests already written and that you write new tests if you've added a significantly new feature that isn't covered by previous tests (new tests aren't needed for this PR). You can check if your PR passes the current tests by running pytest tests in the main "Eureka" folder (you may need to pip install pytest if you haven't already installed it). Pytest also picks up on one of the same errors flake8 found which is that there is now an undefined variable ch_number in the function save_fit inside the fitters.py file.
Finally, also note that in the docstrings for functions, there needs to be a blank line after "History:" otherwise the docs won't be properly formatted (annoying, I know).

Alright, my latest commit should have fixed all of these. Let me know if anything else is needed!

There are still a few warnings/errors with flake8 (use the command I gave above to see them).
There are also now two new errors related to stage 6 with the pytest command. For NIRCam, the following line in the parse_s5_saves function throws an error whose message suggests that fitted_values has no such element "50th":
medians = np.array(fitted_values["50th"])

For the MIRI test, the np.percentile call on line 375 of s6_spectra.py (also in the same parse_s5_saves function) gives an index error that I'm not quite sure the cause of. Since these percentiles are already saved in your csv file though, why not just use those rather than recomputing them?

There are still a few warnings/errors with flake8 (use the command I gave above to see them).
There are also now two new errors related to stage 6 with the pytest command. For NIRCam, the following line in the parse_s5_saves function throws an error whose message suggests that fitted_values has no such element "50th":
medians = np.array(fitted_values["50th"])

For the MIRI test, the np.percentile call on line 375 of s6_spectra.py (also in the same parse_s5_saves function) gives an index error that I'm not quite sure the cause of. Since these percentiles are already saved in your csv file though, why not just use those rather than recomputing them?

For your first point, I fixed all the remaining flake8 issues except the following:
./eureka/S3_data_reduction/background.py:338:41: F821 undefined name 'mask2'
./tests/test_MIRI.py:12:5: F401 'eureka.S1_detector_processing.s1_process as s1' imported but unused

which are not in the files I touched for this PR - but that I wanted to report.
For the NIRCam test, the issue was happening because in the case of a lsq fit, there are no percentiles saved (because there are no samples to draw percentiles from). I added a statement so that it uses the lsq fit result in that case.
For the MIRI test, I implemented the use of the percentiles recorded in the csv file.
I have run the flake8 check and pytest commands again with no issues, but please let me know if there is anything else I can do to make this merge easier :)  + have a good weekend!

I just realized I'm able to push changes to your PR (I couldn't before, but I was using a different method which didn't work since I don't have write privs for your fork), so I made the tiny tweaks that I'd requested. As long as you're happy with the edits I made, you can now click merge below


Correction for raw affiliation Inserm U1111, CNRS UMR 5308, Inserm/CNRS/Université Claude Bernard, Lyon, France

Correction needed for raw affiliation Inserm U1111, CNRS UMR 5308, Inserm/CNRS/Université Claude Bernard, Lyon, France
raw_affiliation_name: Inserm U1111, CNRS UMR 5308, Inserm/CNRS/Université Claude Bernard, Lyon, France
new_rors: 029brtt94;02vjkv261;02feahw73;059sz6q14
previous_rors: 029brtt94;02vjkv261;02feahw73
works_examples: W2443136200
contact: 570b1d820f102d4d93008518380af691:5edcb9789012fbb2c85b695fb101 @ ens-lyon.fr

This issue was accepted and ingested by the OpenAlex team on 2024-12-19. The new affiliations should be visible within the next 7 days.


[nrf fromtree] tests: boards: nrf: coresight_stm: Add performance tes…

…t for STM
Extend STM testing with verification whether STM benchmark results are below predefined thresholds.
Set thresholds arbitrarily.
Add tests that allow code coverage calculation.
Upstream PR: https://github.com/zephyrproject-rtos/zephyr/pull/79000
(cherry picked from commit 29208fe7c45c43997c5099718958f3535f33465b)
manifest-pr-skip

@nrfconnect/ncs-test-leads : Please review.


Use revaultd v0.3.0 common::config

Preparatory work for the revaultd lib integration

self-ACK  e62b8afa88bf26e2998a11748d3868f27976a9a7


Bug fix compile fail on unused value

Proposed changes
error: variable 'nullcount' set but not used [-Werror,-Wunused-but-set-variable]
int nullcount = 0;
Problem summary
Describe your changes.
Checklist(Required)

[ ] Does it affect the original behavior
[ ] Has unit tests been added
[ ] Has document been added or modified
[ ] Does it need to update dependencies
[ ] Is this PR support rollback (If NO, please explain WHY)

Further comments
If this is a relatively large or complex change, kick off the discussion at<EMAIL_ADDRESS>by explaining why you chose the solution you did and what alternatives you considered, etc...

run buildall


ncu-ci: command to start CI for PRs

Done. I also made some changes so that if we fail to start the CI for any reason, the process exits with status code != 0 :)

I think this deserves a @nodejs/build ping to double-check if we're not doing anything we shouldn't in the Jenkins calls (for example, using a user token instead of the token defined in the Job configuration).

Added some tests. Would be great if we could include this in the next release, as it is required for the Request CI Queue on nodejs/node :)

I might not have all the context i might need as i'm not a member of @nodejs/build, but if no one else comments then i think we may be fine to go ahead and merge?

That's what I'm thinking, yes. If something with this approach is not right/good, we can always improve on a future version (or remove if needed)

I'm pretty sure @nodejs/jenkins-admins is a subset of @nodejs/build, but uh, pinging them anyway....

If folks want to help review but don't want/have time to dabble into ncu code, the main question is: is this the right/best way to start a Jenkins job via API? Below is a simplified version of the flow implemented in the PR:
// this code probably doesn't work, but it shows
// which calls the PR makes to start a job via
// Jenkins API

// Use the token generated on Jenkins interface to
// authenticate.
const username = '***';
const personalJenkinsToken = '***';
const auth = Buffer
  .from(`${username}:${personalJenkinsToken}`)
  .toString('base64')
const headers = { Authorization: `Basic ${auth}` };

// Get breadcrumb for CSRF validation
const { crumb } = await (await fetch('https://ci.nodejs.org/crumbIssuer/api/json', { headers }));

// `/job/node-pull-request/build` expects a multipart
// request, so use FormData to generate the payload
const payload = new FormData();
payload.append('json', JSON.stringify({
  parameter: [
    { name: 'CERTIFY_SAFE', value: 'on' },
    { name: 'TARGET_GITHUB_ORG', value: 'nodejs' },
    { name: 'TARGET_REPO_NAME', value: 'node' },
    { name: 'PR_ID', value: 123456 },
    { name: 'REBASE_ONTO', value: '<pr base branch>' },
    { name: 'DESCRIPTION_SETTER_DESCRIPTION', value: '' }
  ]
}));

// Send POST with breadcrump + auth on headers, and
// FormData payload on body
await fetch('https://ci.nodejs.org/job/node-pull-request/build', {
  method: 'POST',
  headers: {
    'Jenkins-Crumb': crumb,
    ...headers
  },
  body: payload
});

Going to merge this just so we can move forward with it :) I think if there are any changes to the flow start we can update as needed!

Can we get a new release with this included? :)
@nodejs/node-core-utils

@mmarchini https://github.com/nodejs/node-core-utils/releases/tag/v1.23.0 🌟

found a bug which doesn't prevent from using the tool, but if the user doens't have permission to start CI it will show as if everything worked fine. I already have a fix locally and will send a PR soon.


Introduce dynamic projectile velocity

Check out the engine function 0x5B66D0 void __thiscall zCRigidBody::SetVelocity(class zVEC3 const &) (5990096). Either utilize it or, if inside there is only a property set, set the property directly. Add a config function like freeAimScaleInitialDamage, in which a custom velocity can be set dynamically.

The config function could look something like this. Estimates on m/s should be provided.
/*
 * This function is called at the point of shooting a bow or a crossbow. The return value scales the velocity of the
 * projectile in percent, where 0 is slow and 100 is the highest velocity allowed.
 */
func int freeAimGetVelocity(var C_Item weapon, var int talent) {
    // Check if bow or crossbow with (weapon.flags & ITEM_BOW) or (weapon.flags & ITEM_CROSSBOW)
    return 50; // 50% corresponds to Gothic's default velocity
};

The velocity here is a vector. Most likely, this is more of a direction, where the actual velocity is not directly evident but encoded in the norm of the vector (speculation). This would not be the best option. However, peeking inside of the engine function might give some insight on the involved class variables.

The velocity is calculated from the mass of the projectile (`0x5B66D0 void __thiscall zCRigidBody::SetVelocity´). Adjusting the mass when setting up the projectile does not work, however, since the velocity is continuously computed from the mass over and over again. Thus, its effect wears off and does not have impact on the velocity.


Kernel Native mode need e2e test

What would you like to be added:
Kernel Native mode need e2e test,
There are currently no end-to-end tests for the kernel-native mode in the repository, making the functionality modifications for this mode very fragile and prone to issues.
There is an urgent need to add restart and large-scale end-to-end test cases.
Why is this needed:

/assigned lec-bit

Hi, I’d like to work on this issue as a first-time contributor—could you confirm if it’s available and share any guidelines?

/assign

@sureshkrishnan-v This task may be a little tricky, because kernel native mode depend on enhanced kernel.  @lec-bit can you provide some guidance on how to run it

Are we going to include guidance on deploying kernel-Native Mode kmesh in the quick-start？

thank you as i am new to open source contribution it would be great if I get help to do this task

@sureshkrishnan-v   Please read docs/kmesh_use_enhanced_kernel.md to get how to run kernel-native mode first.

@sureshkrishnan-v Have you started developing e2e tests yet?and,enhanced kernel?


Issue #35

I know that you have already merged and released this, but I wanted to thank you for adding this feature so fast. 🙏 ❤


Show description for boolean fields

Reasons for making this change
A fix for #435 . I chose to present the description in a similar way like the other fields, with an extra text above the input. The suggested title looked a bit "strange" since it was the only input with that kind of behavior and it would probably have issues in a mobile view.
Checklist

[ ] I'm updating documentation

[ ] I've checked the rendering of the Markdown text I've added
[ ] If I'm adding a new section, I've updated the Table of Content

[x] I'm adding or updating code

[ ] I've added and/or updated tests
[ ] I've updated docs if needed

[x] I'm adding a new feature

[x] I've updated the playground with an example use of the feature

Thanks for the feedback. Should be good to go now!

Thanks!

Released in v0.44.0.


Home Page

Can I work on it?

I can Work on this, please issue me this

@smaranjitghose Sir, I want to work on this:
Currently the flipping card animations look great on the homepage but it is a UX bug for devices on which we  can't hover there is no way for user to know if content exists behind the card until the click on this ( since hover converts to click on such devices)
I want to write advanced media queries to judge if a device is (touch based or supports hover or not) and render these components differently on different devices!!!
On devices on which there is no hover I'll bring the backside of the side to the bottom , so they appear as a single card !!!
@smaranjitghose Sir, Do you think it's needed . If yes, I'd like to this is challenge!

Cleaning -  Removing

Inline CSS from homepage
Deprecated tags like font, center etc.

@ashishnagpal2498 Please be more specific with some Screenshots

@smaranjitghose , use of font as tag. using inline CSS like here -

Hi, I would like to work on these issues on the home/index page.

Add animation to the image slider behind "welcome to GirlsScript Chennai"

rectify the flipped card error. See image.

remove the extra space

add subheading to the following icons:

Sure, go ahead with the changes

Can someone make the home carousel more smoother as it takes really long time to load?
either use optimized images or use a different code for carousel.

Can someone make the home carousel more smoother as it takes really long time to load?
either use optimized images or use a different code for carousel.

@anushbhatia  Sir may I do this?

@abhu-A-J  coordinate with @raniyer on this issue then so that there are no merge conflicts

@smaranjitghose Email id validation required for the input field in the footer.
Can I work on this ?

Ya, please improve on those Shreya


Moves the example app to the Carthage project

Cocoapods is no longer supported. Using new name CGMBLEKit
This is based in part on the work @thebookins did in #68. Thanks!

👍


Use keyless authentication for the release

Use<EMAIL_ADDRESS>for authenticating keyless with OIDC to the AWS account.

I need to do some additional testing.
I'm not fully confident that it will work because of
https://github.com/elastic/apm-aws-lambda/blob/d17142afa3afdf9f771a104ade8e1306e537a6fb/.goreleaser.yaml#L89-L90
Need to check if I also need to pass AWS_SESSION_TOKEN and/or AWS_SECURITY_TOKEN

Added AWS_SESSION_TOKEN
because I saw that the aws/auth action adds the AWS_SESSION_TOKEN as an environment variable.


How to convert a model with .pt tail

Question
I'm new to this area,I'm woundering how to convert a model with .pt tail?thanks a lot

Hi there 👋 Can your model be loaded with Hugging Face transformers? If so, you can convert the model to ONNX with Optimum, so that it can be run with transformers.js.
Please let me know whether your model is on the Hugging Face Hub.


set downsampling factors in the cli

I noticed that the default behavior of the cli appears to be anisotropic downsampling, e.g. x and y dimensions get downsampled but not z. Is there a way to change this via the cli? I have isotropic data, and so I want to downsample it isotropically.

The default behavior is to pick downsampling facts that will result in pixel sizes that will be more uniform in all directions.
Yes, we could add a CLI parameter to specify scale factors by parsing an argument and passing through following:
https://github.com/thewtex/ngff-zarr/blob/4a154994a85e2d222999bb3e5d67ab96f5060244/ngff_zarr/to_multiscales.py#L219-L223
This would make it easy to specify a desired isotropic spacing.


Refactor play track

Description
The main purpose of this pull request is to enhance the process of playing a track.
Test Plan
N/A
Motivation and Context
The general functionality of the app needs to be improved to be leaner and more performant overall. By refactoring not only the PlayTrack async thunk, but as well as the containers where the thunk is used, the process of playing a track can be improved upon.
How Has This Been Tested?
I've manually tested the performance of the app by going through the containers in question and seeing how the new implementation is performing.
Types of Changes

Documentation
Bug fix
[x] New feature
Breaking change

Checklist

[x] My code follows the code style of this project

 Play track thunk


Implement gzip compression for server side

Since gzip enables huge performance benefits for serving assets, we should implement it as an express middleware when building for production.

https://github.com/aickin/shrink-ray


Nest.js + MikroORM + Nestia

Hi Samchon,
This is not really a question, more a discussion. I wanted to share a repository I created with a basic example of Nest.js + MikroORM + Nestia.
I saw that you were supporting Prisma and wanted to try Nestia with MikroORM. It's important for me and my teams as we are not using Prisma at all.
For now, it's an initial draft and I will continue to explore how to better integrate Nestia in this setup. For example, I am not so sure how you process the @hidden comments. Is that only available for Prisma? Also, I feel like there is a bit of a duplication between MikroORM serialization and Nestia serialization.
I still need to integrate Swagger and the SDK, but you can already have a look, especially how it works with MikroORM, if this is something that interests you.
Otherwise, not much to see here. 😉
Thanks for your work on Nestia.

Nestia is just a helper library for SDK and Swagger generation. ORM is not a matter. I've not used MikoORM at all, but there had not been any problem when I used TypeORM with Nestia.
About the @hidden tag, it is just hiding the target property in the Swagger document. It does not affect to the ORM at all, except you've seen the prisma-markdown library.

If your project becomes stable, it would better to add a example chapter and add this one to there.


Shared memory for containers

Hello, would it be possible to specify some IPC mechanisims for containers? Something like --ipc switch to docker does. Basically I need more than one container to have access to a shared memory.

Hi. I'm curious about your use case. Why do you need containers to share memory? I've always considered the idea of share-nothing for containers, to give more flexibility when deploying them.

Alternatively, you could run multiple apps within a single container, and they would share an IPC namespace.

I also seeked complete isolation from containers, however appliance is a good use case. We have a box with high perfomance computing card, this card is represented as a special device and is using DMA for high memory bandwidth (processing 100gb/s of traffic), client applications (running and developed in a container) need to have low latency and high performance access to this area.
Since our engine and client app is developed separately, there should be isolation between those two. I need to guarantee isolation per app (processes, memory and such), otherwise badly written client software can affect our engine.

@stereoit similar to my comment in #99 , I am not really sure this should go directly in the spec since it is a bit platform specific.
I think what I would want is to see an example of this working in an implementation of the spec (either Rocket, for example, or an alternative implementation), and then consider "promoting" it to the spec after feeling that out and giving it some more thought. What do you think?

Given the fact this was also implemented quite recently in docker I think
the proposed solution is OK. Of course it would be cool to see this in rkt
implemented. While reading the Container Runtime Manifest Schema I did
notice it supports multiple apps within one container, which would probably
somehow worked fine for me as well, but this is completely new to me and I
did not find any more documentaion or blogposts on this.
On Mon, Jan 12, 2015 at 9:35 PM, Jonathan Boulle<EMAIL_ADDRESS>wrote:

@stereoit https://github.com/stereoit similar to my comment in #99
https://github.com/appc/spec/issues/99 , I am not really sure this
should go directly in the spec since it is a bit platform specific.
I think what I would want is to see an example of this working in an
implementation of the spec (either Rocket, for example, or an alternative
implementation), and then consider "promoting" it to the spec after feeling
that out and giving it some more thought. What do you think?
—
Reply to this email directly or view it on GitHub
https://github.com/appc/spec/issues/100#issuecomment-69640821.

@jonboulle  +1000 on  your suggestion to explore first to what extend a new requirement can be solved with the current spec, possibly experimenting with alternative implementation, than cluttering the spec with any possible use case. I hope the group can adhere to this philosophy.

I am going to close this for now. @stereoit please feel free to file a feature request against Rocket if you would like to see this there. Also, both this project and rocket are still very young, so documentation and blogposts are a bit sparse, but we're hard at work improving that - any contributions or suggestions for specific docs would be welcome!


Fix persistent cache errors caused by Linux keyring permissions

This prevents the errors that made testing the Linux keyring storage implementation impossible in CI. These errors were caused by the go test process having permission to add keys to the user keyring but not to read those keys or find them in a search. This weird situation arose because the process was accessing the user keyring as its owner, not as a possessor, and the owner has lesser privilege. (Ownership doesn't imply possession in the keyring permissions model, I guess to enable processes running as the same UID to protect secrets from each other.) The fix is to link the session keyring possessed by the process to the user keyring, transitively granting the process possession of keys linked to the user keyring. Most systems do this automatically on login but apparently our CI runners either do not, or they replace the session keyring created at login before running go test. If this all seems too complicated to you, I agree 😅

go

You have elderly relatives who own real stores. It closes and opens in the morning, and the key and copy it into the watch chain. He sleeps with it, washes himself, and takes a bath with it. Because we take you to the picture in focus 😂👌🏾


Page Orientation

I can't find a function to adjust the PDF page orientation. Like portrait orientation or landscape orientation.

No such API till now, will add it to todo.


feat(wallet): add support for ecosystem wallets

This PR adds a new wallet type called ecosystemWallet to the thirdweb SDK.
TL;DR

New ecosystemWallet added to thirdweb SDK.

What changed?

wallets.ts: Added new export for ecosystemWallet
ConnectButtonProps.ts: Adjusted types to account for new wallet.
create-wallet.ts: Added new case for handling ecosystemWallet in the createWallet function.
types.ts: Defined new types for ecosystemWallet.
Updated various internal files to integrate ecosystemWallet functionality.

How to test?

Ensure the ecosystemWallet can be created and connected using different account types (smart account, direct connection, etc.)
Test the new integratorId feature, which allows integration with a designated ecosystem.
Validate that autoConnect functionality works as expected with ecosystemWallet.

Why make this change?
This change aims to facilitate the integration of third-party ecosystem wallets into the thirdweb SDK, improving the overall flexibility and usability of the SDK for different blockchain ecosystems.

PR-Codex overview
This PR adds support for ecosystemWallet and partnerId in the Thirdweb SDK, deprecates smartWallet, and updates various wallet-related components.
Detailed summary

Adds ecosystemWallet to Thirdweb SDK
Introduces partnerId option for ecosystem wallet connections
Deprecates smartWallet
Updates multiple wallet-related components and types

The following files were skipped due to too many changes: packages/thirdweb/src/wallets/in-app/core/authentication/index.ts, packages/thirdweb/src/wallets/in-app/core/wallet/index.ts, packages/thirdweb/src/wallets/in-app/web/lib/in-app-account.ts, packages/thirdweb/src/wallets/create-wallet.ts

✨ Ask PR-Codex anything about this PR by commenting with /codex {your question}

#3270  👈
main

This stack of pull requests is managed by Graphite. Learn more about stacking.


It is mandatory to assign licences to guest users?

The statement
'B2B guest user licensing is automatically calculated' make me think that it is not mandatory to assign licenses to guest users, is it correct?
I understand that products like PowerApps require licenses to be explicitly assigned to users, is it the same for AADP licences to use MFA for guests?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: d9de7df3-1669-e58d-ae57-bdc8993ac35f
Version Independent ID: 3a1d7ec3-b46f-3c87-2dc6-846cc815a530
Content: B2B collaboration licensing guidance - Azure Active Directory
Content Source: articles/active-directory/b2b/licensing-guidance.md
Service: active-directory
Sub-service: b2b
GitHub Login: @msmimart
Microsoft Alias: mimart

@AlexCesarini
Thanks for your feedback! We will investigate and update as appropriate.

Hi @AlexCesarini, For any paid Azure AD features, you can invite up to five guest users for each Azure AD edition license that you own. This includes Azure MFA. hhttps://docs.microsoft.com/en-us/azure/active-directory/b2b/licensing-guidance#
Hope this helps! If you have further questions feel free to reach out on Microsoft Q&A. https://docs.microsoft.com/answers/topics/azure-active-directory.html?filter=unanswered

I actually asked about assigning licences manually to guests, this does not answer my question.


Use path unit to monitor the change in identity file to restart maestro

With path units, you can monitor files and directories for events and use them to execute service units. Added path file to monitor identity.json and call service unit which will restart maestro service. Once the gateway has been claimed, maestro needs to be restarted to consume the new identity and create config files for other gateway services.

@costanic Currently maestro is responsible to manage devicejs, devicedb and other protocol translators services. So when maestro restarts, they will all restart.
I think we have do the same for relay-term service. Should I add those files to this PR?
deviceos-wd service is not affected by identity change.
I think those are all the services dependent on identity change.

relay-term doesn't necessarily need to be included in this PR, it can certainly be in a different PR.  I just wanted to ask the question to make sure we didn't forget some services that are also dependent on identity.json.

@costanic Ya I dont think apart from maestro and relay-term there is another service which is dependent on identity.json. I will create another PR to bring in watchers for relay-term service.

@ygoyal18 Would this link be a good reference for the feature?
https://www.freedesktop.org/software/systemd/man/systemd.path.html


Cannot get the consistent results

Trial 01
array([[0.85714286],
       [0.85715105],
       [1.        ],
       [1.00000015],
       [1.        ]])

Trial 02
array([[0.85714286],
       [0.85714286],
       [1.00000046],
       [1.        ],
       [1.        ]])

I get the different result from different trials because of The random setup x0 = np.random.rand(d0) - 0.5?
I can set the random seed, but does this fact show training is not converged?

 The problem is the initial set is randomized without a fixed seed.
        for unit in self.unit_:
            # weights
            np.random.seed(123)
            x0 = np.random.rand(d0) - 0.5
            x0 = fmin_slsqp(self.__target, x0, f_ieqcons=self.__constraints, args=(unit,))
            # unroll weights

here is the solution.


Consumer stops buying

Sometimes the consumer stops buying products.
I suspect that a specific buying behavior crashes if it is chosen. In that case, the consumer fails silently.
The consumer buys again, if it is stopped and started.
A temporary workaround is to only use buying behaviors that are known to work (cheapest, second cheapest).
I had the same problem when defining a new buying behavior without an implementation.
The consumer fails silently when trying to call the non-existing function.
I'm on the docker setup on Ubuntu.

For executing buying behaviors, Ruby's thread function is used for multithreading. If there is an exception in this thread, it would be raised. For debugging, you may comment that out in line 57 and see if an error is thrown.

Is that now fixed?

This is still an issue.
It's now possible to see a stack trace of the error when running the consumer in debug mode.
This will help finding the bug.


Build from sources with Cuda 9.0 cudnn 7.1 on Window 10.

Hi, everyone,
Has anyone tried to compile mxnet from sources with Cuda 9.0 cudnn 7.1 on Window 10?
I used VS2015, Cuda 9.0, cudnn7.1.
When I use CMake to generate sln file for VS. I noticed that it print that:
CUDA detected: 9.0 Found cuDNN (include: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v9.0/include, library: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v9.0/lib/x64/cudnn.lib) Added CUDA NVCC flags for: sm_90
When I compile the generated vs project, the compiler told me that compute_90 was not  nvcc supported.
I know that the maximum compute_xx should be compute_70.
But I don'k know why mx-net will "Added CUDA NVCC flags for compute_90".
Anyone know how to fix this?

Can you grep around to see what is trying to use "compute_90" obviously that's not supported. Is it in the visual studio build files?

@larroy
Thanks for your reply.
As I wrote in the question, the CMake logged information that "Added CUDA NVCC flags for: sm_90". This message was raised from "./3rdarty/mshadow/cmake/Cuda.cmake"(Line around 285-289).
# setting nvcc arch flags
mshadow_select_nvcc_arch_flags(NVCC_FLAGS_EXTRA)
list(APPEND CUDA_NVCC_FLAGS ${NVCC_FLAGS_EXTRA})
message(STATUS "Added CUDA NVCC flags for: ${NVCC_FLAGS_EXTRA_readable}")
It's ok to build the sln file using CMake. But when I compiled the "mxnet project" in VS, it reported errors:
Building NVCC (Device) object
CMakeFiles/cuda_compile_1.dir/src/common/Release/cuda_compile_1_generated_random_generator.cu.obj
3>  nvcc fatal   : Unsupported gpu architecture 'compute_90'
3>  CMake Error at cuda_compile_1_generated_random_generator.cu.obj.Release.cmake:222 (message):
3>    Error generating
3> E:/caffe_test/mxnet_20180430/build/CMakeFiles/cuda_compile_1.dir/src/common/Release/cuda_compile_1_generated_random_generator.cu.obj
3>
Thus, I think there might be some problems  in the function:
mshadow_select_nvcc_arch_flags
(But I not so familiar with CMake language, so I don't know how to find the problem and fix it)

@larroy @nswamy
Finally I find an alternative way to skip this problem.
Step1:  using CMake-gui do the normal configure process;
Step2:  find the CUDA option, an select the "CUDA_ARCH_NAME" to "Manual"; then click configuration;
Step3: change the new appear( usually in red in the GUI) to what you want. (for example: sm_70); then click configuration and generate;
Step4: compile the generated sln file using VS IDE.

Cuda.cmake is used to detect GPU compute capability by calling nvcc command like "${CUDA_NVCC_EXECUTABLE} --run  ${__cufile}" , then regular expression is caled in cmake to extract cuda compute capability string: string(REGEX MATCH "([1-9].[0-9])" __nvcc_out "${__nvcc_out}")
In my computer with cuda9.0, what the command output is：c:\program files\nvidia gpu computing toolkit\cuda\v9.0\include\crt\math_functions.h : warning C4819: .......
6.1
Here comes the question, __nvcc_out is set to be 9.0 rather than 6.1(which is the right one we want), since 9.0 is wrongly matched, and that's why we saw "sm_90". Now the solution is clear, add "-w" arguments after nvcc command to depress warnings:  ${CUDA_NVCC_EXECUTABLE} --run  ${__cufile} "-w" ..., or change the regular expression to ([^a-zA-Z][1-9]\\.[0-9]).

@vipandrew  Thanks for your kindly reply. This will help a lot.

@vipandrew @chaoer  good point, can we create a PR for that?


Cargo build error for Rust SDK

Hi all,
When I tried to cargo build /teaclave/sdk/rust/, I met the following error:
error: failed to run custom build command for `teaclave_proto v0.1.0 (/teaclave/services/proto)`
Caused by:
  process didn't exit successfully: `/teaclave/sdk/rust/target/debug/build/teaclave_proto-1f6374e59418fb11/build-script-build` (exit code: 101)

--- stderr
thread 'main' panicked at 'stdout: "error: failed to download `ansi_term v0.11.0`\n\nCaused by:\n  can\'t make HTTP request in the offline mode\n", stderr: "error: failed to download `ansi_term v0.11.0`\n\nCaused by:\n  can\'t make HTTP request in the offline mode\n"', /teaclave/services/proto/build.rs:76:9
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

warning: build failed, waiting for other jobs to finish...
error: build failed

The problem can be solved by commenting cargo build option "--offline"  in /services/proto/build.rs. I guess there are some new packets needed by building the sdk. Maybe a update is needed here.
Thanks

The problem can be solved by commenting cargo build option "--offline"  in /services/proto/build.rs. I guess there are some new packets needed by building the sdk. Maybe a update is needed here.
Thanks

Thanks for reporting this. No, this is not caused by the needs of new crates.
This is because all of our dependencies for the frameworks, SDKs, cli, etc. are vendored for offline building. However, this is managed by the build system (i.e., cmake). If you want to compile a crate directly in side the directory, there may be some compilation errors.
In your case, I'm thinking about the solution. Maybe, putting a Cargo.lock should work.

Thanks for reporting this. No, this is not caused by the needs of new crates.
This is because all of our dependencies for the frameworks, SDKs, cli, etc. are vendored for offline building. However, this is managed by the build system (i.e., cmake). If you want to compile a crate directly in side the directory, there may be some compilation errors.
In your case, I'm thinking about the solution. Maybe, putting a Cargo.lock should work.


some adultswing.com download not working

youtube-dl --verbose http://www.adultswim.com/videos/rick-and-morty/season-3-exclusive-san-diego-comic-con-2016/
[debug] System config: []
[debug] User config: []
[debug] Command-line args: [u'--verbose', u'http://www.adultswim.com/videos/rick-and-morty/season-3-exclusive-san-diego-comic-con-2016/']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2016.11.18
[debug] Python version 2.7.9 - Linux-4.2.0-0.bpo.1-amd64-x86_64-with-debian-8.6
[debug] exe versions: ffmpeg 2.8.8, ffprobe 2.8.8, rtmpdump 2.4
[debug] Proxy map: {}
[AdultSwim] season-3-exclusive-san-diego-comic-con-2016: Downloading webpage
ERROR: Unable to extract bootstraped data; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Traceback (most recent call last):
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 694, in extract_info
    ie_result = ie.extract(url)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 357, in extract
    return self._real_extract(url)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/adultswim.py", line 152, in _real_extract
    r'var bootstrappedData = ({.*});', webpage, 'bootstraped data'), episode_path)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 660, in _search_regex
    raise RegexNotFoundError('Unable to extract %s' % _name)
RegexNotFoundError: Unable to extract bootstraped data; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.

Duplicate of #11042