sensor.trash_papier_en_karton missing

Hi there,
I'm missing a sensor this last week or so. Do you know what can be the cause.
entity not available: sensor.trash_papier_en_karton

platform: afvalwijzer
provider: mijnafvalwijzer

The other 3 are working correct.
I'm using Afvalwijzer 4.2.0
On Hassio Home Assistant Core 0.109.3
Hope you know what can be the cause.

Kind regards,
Rob

Hi Rob,
If you can provide me your zipcode and housenumber I can verify some things. Did you check if the sensor perhaps has been renamed?
Kinds regards,
Bram

I did check, there are only 3 sensors available.
'GFT', 'Plastic' and 'Restafval'.
zipcode: 3911CX
housenumber: 178

Here the available entities

It looks like the data is not available from mijnafvalwijzer (maybe not correctly provided by your community). I only receive the following data:
[{'key': 'pmd', 'value': '07-05-2020', 'days_remaining': 2}, {'key': 'gft', 'value': '07-05-2020', 'days_remaining': 2}, {'key': 'restafval', 'value': '14-05-2020', 'days_remaining': 9}]
[{'key': 'first_next_in_days', 'value': 2}]
[{'key': 'today', 'value': 'None'}]
[{'key': 'tomorrow', 'value': 'None'}]
[{'key': 'day_after_tomorrow', 'value': 'pmd, gft'}, {'key': 'day_after_tomorrow', 'value': 'pmd, gft'}]
[{'key': 'first_next_item', 'value': 'pmd'}]
[{'key': 'first_next_date', 'value': '07-05-2020'}]
['gft', 'pmd', 'restafval']

This is the raw data from the website itself, also no paper found there:
"ophaaldagen":{"response":"OK","data":[{"nameType":"gft","type":"gft","date":"2020-01-02"},{"nameType":"pmd","type":"pmd","date":"2020-01-02"},{"nameType":"restafval","type":"restafval","date":"2020-01-09"},{"nameType":"pmd","type":"pmd","date":"2020-01-15"},{"nameType":"gft","type":"gft","date":"2020-01-16"},{"nameType":"restafval","type":"restafval","date":"2020-01-23"},{"nameType":"gft","type":"gft","date":"2020-01-30"},{"nameType":"pmd","type":"pmd","date":"2020-02-06"},{"nameType":"restafval","type":"restafval","date":"2020-02-06"},{"nameType":"gft","type":"gft","date":"2020-02-13"},{"nameType":"pmd","type":"pmd","date":"2020-02-19"},{"nameType":"restafval","type":"restafval","date":"2020-02-20"},{"nameType":"gft","type":"gft","date":"2020-02-27"},{"nameType":"restafval","type":"restafval","date":"2020-03-05"},{"nameType":"pmd","type":"pmd","date":"2020-03-05"},{"nameType":"gft","type":"gft","date":"2020-03-12"},{"nameType":"pmd","type":"pmd","date":"2020-03-18"},{"nameType":"restafval","type":"restafval","date":"2020-03-19"},{"nameType":"gft","type":"gft","date":"2020-03-26"},{"nameType":"pmd","type":"pmd","date":"2020-04-02"},{"nameType":"restafval","type":"restafval","date":"2020-04-02"},{"nameType":"gft","type":"gft","date":"2020-04-09"},{"nameType":"pmd","type":"pmd","date":"2020-04-15"},{"nameType":"restafval","type":"restafval","date":"2020-04-16"},{"nameType":"gft","type":"gft","date":"2020-04-23"},{"nameType":"restafval","type":"restafval","date":"2020-04-30"},{"nameType":"pmd","type":"pmd","date":"2020-05-07"},{"nameType":"gft","type":"gft","date":"2020-05-07"},{"nameType":"restafval","type":"restafval","date":"2020-05-14"},{"nameType":"pmd","type":"pmd","date":"2020-05-20"},{"nameType":"gft","type":"gft","date":"2020-05-21"},{"nameType":"restafval","type":"restafval","date":"2020-05-28"},{"nameType":"pmd","type":"pmd","date":"2020-06-04"},{"nameType":"gft","type":"gft","date":"2020-06-04"},{"nameType":"restafval","type":"restafval","date":"2020-06-11"},{"nameType":"pmd","type":"pmd","date":"2020-06-17"},{"nameType":"gft","type":"gft","date":"2020-06-18"},{"nameType":"restafval","type":"restafval","date":"2020-06-25"},{"nameType":"gft","type":"gft","date":"2020-07-02"},{"nameType":"pmd","type":"pmd","date":"2020-07-02"},{"nameType":"restafval","type":"restafval","date":"2020-07-09"},{"nameType":"pmd","type":"pmd","date":"2020-07-15"},{"nameType":"gft","type":"gft","date":"2020-07-16"},{"nameType":"restafval","type":"restafval","date":"2020-07-23"},{"nameType":"gft","type":"gft","date":"2020-07-30"},{"nameType":"restafval","type":"restafval","date":"2020-08-06"},{"nameType":"pmd","type":"pmd","date":"2020-08-06"},{"nameType":"gft","type":"gft","date":"2020-08-13"},{"nameType":"pmd","type":"pmd","date":"2020-08-19"},{"nameType":"restafval","type":"restafval","date":"2020-08-20"},{"nameType":"gft","type":"gft","date":"2020-08-27"},{"nameType":"pmd","type":"pmd","date":"2020-09-03"},{"nameType":"restafval","type":"restafval","date":"2020-09-03"},{"nameType":"gft","type":"gft","date":"2020-09-10"},{"nameType":"pmd","type":"pmd","date":"2020-09-16"},{"nameType":"restafval","type":"restafval","date":"2020-09-17"},{"nameType":"gft","type":"gft","date":"2020-09-24"},{"nameType":"restafval","type":"restafval","date":"2020-10-01"},{"nameType":"gft","type":"gft","date":"2020-10-08"},{"nameType":"pmd","type":"pmd","date":"2020-10-08"},{"nameType":"restafval","type":"restafval","date":"2020-10-15"},{"nameType":"pmd","type":"pmd","date":"2020-10-21"},{"nameType":"gft","type":"gft","date":"2020-10-22"},{"nameType":"restafval","type":"restafval","date":"2020-10-29"},{"nameType":"gft","type":"gft","date":"2020-11-05"},{"nameType":"pmd","type":"pmd","date":"2020-11-05"},{"nameType":"restafval","type":"restafval","date":"2020-11-12"},{"nameType":"pmd","type":"pmd","date":"2020-11-18"},{"nameType":"gft","type":"gft","date":"2020-11-19"},{"nameType":"restafval","type":"restafval","date":"2020-11-26"},{"nameType":"pmd","type":"pmd","date":"2020-12-03"},{"nameType":"gft","type":"gft","date":"2020-12-03"},{"nameType":"restafval","type":"restafval","date":"2020-12-10"},{"nameType":"pmd","type":"pmd","date":"2020-12-16"},{"nameType":"gft","type":"gft","date":"2020-12-17"},{"nameType":"restafval","type":"restafval","date":"2020-12-24"}],"error":false},"ophaaldagenNext":{"response":"NOK","data":[],"error":"No data"}

https://json.mijnafvalwijzer.nl/?method=postcodecheck&postcode=3911CX&huisnummer=178&toevoeging=&langs=nl

So closing this issue, as it isn't related to the component.

Thanks mate, let's wait when it's provided again

You're welcome! Otherwise contact mijnafvalwijzer.nl.
On the website it's also not available: https://mijnafvalwijzer.nl/nl/3911CX/178/


Skip feed Spark partitions on executors running ps tasks

When a python-worker launches a TFManager for ps node, the TFManager doesn't have input queue. If we launch a train/inference operation on the same executor on another python-worker, it will try to connect to the TFManager singleton and access the input queue. It will throw an exception like:
File "./tfspark.zip/com/yahoo/ml/tf/TFManager.py", line 29, in <lambda>
  TFManager.register('get_queue', callable=lambda qname: qdict[qname])
KeyError: 'input'

Please let me know if this patch is appropriate. Thanks.

While this would work, I'd prefer to avoid creating task failures "under normal operation", since they might mask other real failures.  Also, in the optimal case, a PS task should "block" and never receive data feeding tasks.  So, the ideal fix would be to "clean up" any leftover/incorrect state from a failed job (that leads to this symptom)... but at the same time, we need to distinguish between "leftover/incorrect state" and "desired state".

If an executor has more than one core, I think it can run more than one python-worker, so it will not be blocked by a PS task. There is no guarantee that the train func will not run on the executor.

Yes, I believe that's correct, but I think configuring the executors for more than one core might take a bit more vetting of the current code.  That all said, I think I have an idea of how to prevent this error (for the single core case at least)... details pending.  Once we get past that, I think we can revisit the multi-core case.

For the single core case, it is weird since you only have one python-worker running on the executor. How does the manager without input queue to be connected from a worker task?

It's possible because Spark, by default, can spawn a pool of Python workers to handle tasks.  This is the main reason why we use the multiprocessing.Manager and Queue, s.t. we can find the queue regardless of which Python worker we're in.

FYI, I've submitted #37 which tries to detect the cause of the KeyError problem.  Note that this is really just a symptom of incorrect/invalid reservation.  In the current design, we assume that a Spark executor equates to a TF Node.  However, due to Spark's pool of Python workers (even for single-core configs), there is still the possibility that the same executor might reserve two different TF nodes of the cluster_spec.  And since we tie each TFNode to a TFManager via a unique (host, ppid) identifier (where ppid is the pid of the executor's JVM that spawned the Python worker), this can lead to incorrect behavior.

It's possible because Spark, by default, can spawn a pool of Python workers to handle tasks. This is the main reason why we use the multiprocessing.Manager and Queue, s.t. we can find the queue regardless of which Python worker we're in.

Don't you say ps task will block it and never receive data feeding tasks under single core?

Sort of... the best way to explain is to use this modified mnist_spark.py snippet, which tries to run train immediately followed by test, but without a clean shutdown in between.
Under normal operation, the reserve step will run on distinct executors, assigning one TF node per executor.  Note that we have the raise Exception in the original code to help ensure this.  The start step will then start the TF processes, one per-executor, but in the background.  The worker tasks will complete and free up the executors, but the PS nodes will block the task completion by waiting for a control signal.  For RDD feeding, we then use SparkContext's threading capability to spawn a new feeding job, which will not assign any of the data feeding tasks to the still-busy executor (hosting the PS process).  When the data feeding completes, we use the shutdown step to notify the blocked PS task to finally exit, which completes the initial start job.
Now, if we don't invoke the shutdown cleanly (per the gist), the PS task will continue to tie up one executor.  So, if we follow up with another reserve, Spark will now try to assign the N reserve tasks on N-1 nodes.  It will throw various TFManager already started exceptions, but it will eventually complete because of Spark's Python worker pools.  However, we end up with two TF nodes (e.g. worker + ps) assigned to the same executor.  And, in my tests, the PS node reserved last, so it now "owns" the TFManager for that executor.  Subsequently, when we try to start both TF nodes on this problematic executor, it actually starts the worker first (because we actually had the PS nodes wait a bit on startup).  And when we start feeding data into it, it connects to the local TFManager (which now has PS queues only), and fails with the KeyError.
I hope that made sense...

So, if we follow up with another reserve, Spark will now try to assign the N reserve tasks on N-1 nodes. It will throw various TFManager already started exceptions, but it will eventually complete because of Spark's Python worker pools.

I think most of it makes sense except the above. As the executors are configured with only one core, it should only be able to run just one python worker, otherwise your PS task doesn't be able to block the executor, because the executor can spawn another python worker.

OK, I think I see the confusion now...
One core means Spark will only assign one task at a time to that executor.  However, once the task is assigned to the executor, Spark can execute that task on a Python worker from a pool of workers.
So, the "bad" reserve task is retried multiple times (on the executor), and one of those times, it will succeed if on a "fresh" worker from the pool.

Regardless, I think the KeyError is a symptom of a failed reservation, and I'm not sure if this PR will completely solve the problem in those cases.

One core means Spark will only assign one task at a time to that executor. However, once the task is assigned to the executor, Spark can execute that task on a Python worker from a pool of workers.
So, the "bad" reserve task is retried multiple times (on the executor), and one of those times, it will succeed if on a "fresh" worker from the pool.

Spark won't create a pool of workers in advance. It will create a python-worker when requested to run a task involving Python exec. If reuse-python-worker config is enabled, this worker can be put back to a queue to be reuse later (the pool in your word).
If the executor is only with one core, it can run only one task. In this single task, you will have just only one python-worker running. Once it finishes, it is put back to the queue and waits for next task. So the pool has only one python-worker.
Only if your executor is configured with more than one core, so it can run multiple tasks and also python-workers at the same time. Then it is possibly you can choose one "fresh" python-worker from many idle python-workers.

Can you try modifying the mnist example per the gist in your environment?  I'd be curious to see how your environment behaves...

Btw, the error is not only happened when shutdown() is not cleanly called. With the unmodified mnist_spark example, it can happen too.

Can you send me the logs from one of those instances?

Looks like part of issues (if not all) are solved by #37. Close this for now.


Form storage in routable panels

Framework7 version: 5.7.8
Platform and Target: all
Live Link or JSFiddle: https://jsfiddle.net/4gwfcevs/

Describe the bug
When you include a form with form store data inside a routable panel, data isn't stored in local storage. Reason is that the form isn't initialised when the panel is made visible. form.js only initialises form storage on tabs and pageInit, but pageInit is never triggered for routable panels.
To Reproduce
Steps to reproduce the behavior:

Click on right panel
Fill in form
Check local storage (inspect element in desktop browser)
Localstorage is empty.

Expected behavior
Localstorage should contain an entry f7form-my-form
Actual Behavior
Localstorage is empty
Additional context
can be solved by adding (form.js line 309):
panelOpen: function panelOpen(panel) {
    var app = this;
    panel.$el.find('.form-store-data').each((index, formEl) => {
      app.form.storage.init(formEl);
    });
  },
  panelClose: function panelOpen(panel) {
    var app = this;
    panel.$el.find('.form-store-data').each((index, formEl) => {
      app.form.storage.destroy(formEl);
    });
  }

(Or pageinit should be triggered for routable panels ??? )
Probably also happens for routable modals ?

Everything that has kind of "auto initialization" requires View + Page to be initialised. Make sure you form in panel is wrapped with View > Page to work correctly

But as you can see in the fiddle, there is a view and page tag around the form?


test greenboot ci

/greenboot-rs-test-rawhide say-paul/images:remove-old-greenboot-services

/greenboot-rs-test-rawhide say-paul/images:remove-old-greenboot-services

/greenboot-rs-test-rawhide say-paul/images:remove-old-greenboot-services

/greenboot-rs-test-rawhide say-paul/images:remove-old-greenboot-services

/greenboot-rs-test-rawhide yih-redhat/images:image-ci

/greenboot-rs-test-rawhide yih-redhat/images:image-ci

/greenboot-rs-test-rawhide yih-redhat/images:image-ci

/greenboot-rs-test-39 yih-redhat/images:image-ci

Current status:

something wrong with say-paul/images:remove-old-greenboot-services, change to another image, and osbuild-composer can build successfully.
the greenboot rpm built from say-paul/greenboot branch greenboot-rs-continued caused osbuild-composer failed to build image, need to take look.

/greenboot-rs-test-39 say-paul/images:remove-old-greenboot-services

/greenboot-rs-test-39 say-paul/images:remove-old-greenboot-services

/greenboot-rs-test-39 say-paul/images:remove-old-greenboot-services

/greenboot-rs-test-39 say-paul/images:remove-old-greenboot-services

/greenboot-rs-test-39 say-paul/images:remove-old-greenboot-services


Support dtypes other than float in sparse data structures

Would it be easier to work towards supporting floats other than float64 first? The overall enhancement of supporting other kinds of dtypes seems like a major effort that should probably be tackled in smaller steps.
I'm particularly interested in reducing memory usage of dummy variables (i.e. bool) and small-valued counts (e.g. uint8). When sparsifying dataframes that contain such dtypes, being able to convert to float16 rather than float64 would already help a lot.
I've posted a StackOverflow question (and answer) regarding my attempts to achieve this.

@aolieman sparse already supports quite a few dtypes, and more in 0.18.1, see changes here

@jreback thanks for mentioning the 0.18.1 fixes. They should solve some of the issues I encountered, but dtype coercion still occurs with frames. Even if a SparseDataFrame is directly constructed from a SparseSeries that has the desired dtype, the resulting frame always uses float64.
Attempts to construct a one-column frame in 0.18.0 (same result for multiple columns):
In []: dense_series = pd.Series([False]*5 + [True]*3 + [False]*5, dtype='bool', name='b')

In []: dense_df = pd.DataFrame(dense_series)

In []: sparse_df = dense_df.to_sparse(fill_value=False)

In []: sparse_df['b'].dtype
Out[]: dtype('float64')

In []: sparse_series = dense_series.to_sparse(fill_value=False)

In []: sparse_series.dtype
Out[]: dtype('bool')

In []: sparse_df = pd.SparseDataFrame(sparse_series)

In []: sparse_df['b'].dtype
Out[]: dtype('float64')

In []: sparse_df = pd.SparseDataFrame(sparse_series, dtype='bool')

In []: sparse_df['b'].dtype
Out[]: dtype('bool')

In []: sparse_df.info()
------------------------
[traceback omitted]
AttributeError: ("'SingleBlockManager' object has no attribute 'view'", 'occurred at index b')

In []: sparse_df['b'].values
Out[]:
SingleBlockManager
Items: RangeIndex(start=0, stop=13, step=1)
BoolBlock: 13 dtype: bool

My apologies if this is solved in 0.18.1 (which I'm not able to test right now) or if I'm doing it wrong.

well it's an open issue - welcome to have test and such


README: Fix markdown rendering of ALLOWED_REQUEST_CONTENT_TYPE

The default value of ALLOWED_REQUEST_CONTENT_TYPE was not rendered correctly due to some unescaped pipe characters in the markdown table.

Good catch! Thanks for the contribution!


deadband

Can you explain what would be the deadband please.
As i understand it stands for the gap which is close to the bias point and doesnt actually move the motor.
As an example if we set the output limits to -255 , 255 then if the motor can not actually move in between -30 , 30 deadband should recalculate these values and  only use  the values between -30 -255 and 30 255 is that right ?
And i can not see its usage in the code. Not implemented yet ?
Thanks in advance

You're right, it seems I added all the architecture for deadband and then forgot to implement or something. Been a long while since I worked on this codebase, so I forget how this particular piece was supposed to work. However, I think it's supposed to work as you describe...


Italian

https://robertsspaceindustries.com/spectrum/community/SC/forum/50172/thread/localization-italian/6274646
Wolverine@GattoMatto
here is the file. Please add me and MrRevo as reference
https://drive.google.com/file/d/1Fm9KAcOAzK0THj_wAki32ibhAgRmTYG3/view?usp=sharing

I'm trying to download the file but it says it's in the owner's trash...
It would be a great thing to merge the works, even if there would be many lines to double check.

The Italian file is on this page and we also collaborate with them I think?
https://sctranslator.danidomen.com/

I didn't know about it! I immediately propose it to our translators, I imagine it is a machine translation job. We translated many lines manually, let's try to understand how to combine the works!
@Dymerz are we collaborating with them?

Well this repo mentioned on their page^^

I'm checking by phone, in the late afternoon I should have time to check better, I saw that the Italian translations are completely automatically generated.
On the one hand I would hate to lose the work done so far, given that we are translating every single line by hand, on the other I understand that this is a "ready" work even if with several errors.
Merging the two works would require manually checking all the lines translated so far I guess, we'll see what to do

@Anyma6, I'm not collaborating with them at the moment, and I don't know if they are working on any translations on their end. I should reach out to them.
Moreover, manual translations are usually better than AI. If possible, I will merge the two files, prioritizing the human-translated content first. I'm already doing this with the French translation using the Update French action

Finally got the italian file.
We should mention both Wolverine and MrRevo as wished by Wolverine.

Good, I made the mention on the README.md


(yes/no) accepted y as correct answer in artisan migration

Laravel Version: 8.77.1
PHP Version: 8
Database Driver & Version: PosgreSQL

Description:

Is it intentional ? I typed y instead of 'yes' or 'no' and it run the migration... (in phpstorm terminal)

Yeah. It covers both cases. It is assumed "y" stands for "yes" and "n" for "no". Pretty nifty, you see?

it is common behavior on command line when prompted for answers to just reply with y, you can try it with other tools such as yum, apk, rm, etc.

Thanks @humaneguy @bernardwiesner


Web not being correctly disabled.

Describe the bug
Setting
web:
  enabled: false

at the end of  config.yaml is not disabling the web interface.
To Reproduce

Navigate to the web interface in a browser
Expected behavior
No web interface available.
Screenshots

Docker || Kubernetes (please complete the following information):

Docker verdaccio tag: verdaccio/verdaccio   4   a53fabc46052        2 months ago        123MB
Docker commands [e.g. docker pull ...]
Docker Version: Docker version 18.06.1-ce, build e68fc7a215d7133c34aa18e3b72b4a21fd0c6136

Configuration File (cat ~/.config/verdaccio/config.yaml)
storage: /verdaccio/storage
auth:
  htpasswd:
    file: ./htpasswd
uplinks:
  npmjs:
    url: https://registry.npmjs.org/
    maxage: 2d
packages:
  '@my-company/*':
    access: $authenticated
    publish: $authenticated
    unpublish: admin
  '**':
    access: $authenticated
    proxy: npmjs
logs:
  - {type: stdout, format: pretty, level: http}
web:
  enabled: false

Environment information
verdaccio --info:
Environment Info:
System:
OS: Linux 4.14 Alpine Linux undefined
CPU: (2) x64 Intel(R) Xeon(R) Platinum 8175M CPU @ 2.50GHz
Binaries:
Node: 10.16.2 - /usr/local/bin/node
Yarn: 1.17.3 - /usr/local/bin/yarn
npm: 6.9.0 - /usr/local/bin/npm
verdaccio --version:
4.2.1

Works for me, at docker and npm.


Add SearchApi documentloader, tool & podcast QA example

Changes:

Added SearchApi document loader for Web Search Results.
Added SearchApi tool for real-time Google Search queries.
Added "Chat with podcast" example.

Feel free to assign @SebastjanPrachovskij as a main reviewer for any SearchApi PRs. We will be glad to help and support Flowise development.

Usage:

if you can run yarn lint-fix @SebastjanPrachovskij , everything else looks good!

@HenryHengZJ Done, thank you!


Feature/self assesment field

This PR re-implements the self_assesment field, this time for the whole proposal, on the general page.

The field should also be added to the diff pages (proposals/templates/proposals/diff)

Ok boys, I think everything has been updated that needed updating. The only thing I am still unsure about is the conflict in locale/en/LC_MESSAGES/django.po... I sure don't understand what any of it means, and you probably do, so I await your input.


Fixed IndexError when list index != partition number

Hi @exception13 , thanks for the fix.
We'll review the code shortly but in the interim, can I ask you to sign our CLA please? I added the CLA hook yesterday just minutes after you opened this PR and will be updating the repository contributing guidelines shortly. I am hoping that this comment will trigger the hook to run and prompt you with a link.
Let me know if you have any questions about this.

Hi @exception13,
Welcome to the project, and thanks for your contribution! It's great to grow the community and contributor list.
The could you elaborate on the summary slightly so we can understand the issue that is being addressed? It's important that other contributors can understand the mistake or assumption made  to prevent future occurrences elsewhere.
Finally, to document the error and prevent the error from reoccurring. Would you be able to add a simple regression test. One that would have failed before your PR, but now passes.
The CLA bot is having trouble associating your registration. You are showing up slightly differently on the commit author; could you check the email address in the commit is registered with your GH account and the CLA form?
Thanks again,

Great, this looks good. Test failure is just the python 3.7 async/asynchronous change.
Thanks for this!


NavigationAgent2D does not apply radius settings around NavigationPolygonInstance in 3.5 beta 4

Godot version
v3.5.beta4.official [b6968ab06]
System information
Windows 10
Issue description
Hello.
I have a NavigationAgent2D on which I am setting a radius. However, that radius does not seem to be affecting the paths provided by $NavigationAgent.get_next_location() and $NavigationAgent.get_nav_path(). The returned paths are hugging tight to corners and walls, whereas I would expect them to be offset by the radius value.

Steps to reproduce
Download and run the reproduction project. Despite a radius being set in the navigationagent (as confirmed with a print output), it hugs right to corners as if radius was 0.
Minimal reproduction project
Learning 3.zip

Related to https://github.com/godotengine/godot/issues/60354 (possible duplicate)?

Maybe? #60354 seems to show a 3D navmesh correctly baking the safety radius, but then the NavigationAgent3D is traveling outside the navmesh for some reason. #57486 seems to show a NavigationAgent2D responding in apparently suboptimal ways to a NavigationObstacle2D - although it is correctly using the radius to avoid the NavigationObstacle2D.
This issue is a NavigationAgent2D not employing the radius when navigating around a NavigationPolygonInstance. I'm not using any NavigationObstacle2D.
(Not saying they're not related under the hood, just wanted to document what I perceived the differences between the issues to be)

@Calinou I think the milestone for this one should be 3.5.
Regarding the issue itself, it's unrelated to 3D.
The issue here is due to the fundamental problem of 2D navigation in Godot which is: there is no baking stage.
In 3D for example, we have to bake navigation mesh before we can use it. One of the parameters we provide to the baking stage is agent_radius which actually should be called max_agent_radius. It tells the navmesh calculator what is the largest possible agent radius which can be used in the context of this particular navmesh. The navmesh calculator uses it to shrink the navmesh in a way that the largest agent will fit into the original geometry. So e.g. if we have a 10x10 plane and max_agent_radius=2 the resulting navmesh will be shrunk by 2 units in every direction, so it will be 6x6 so that every agent with R<=2 can safely walk 10x10 plane by moving its center within that 6x6 area.
In 2D navigation we have no baking stage and thus the user must take care of polygons himself. This is fine if one is creating navigation polygons from code, but the problem strikes if one uses a tilemap. In that case, one would have to introduce extra tiles with navigation margins which is so cumbersome that I consider it virtually impossible.
The only way to overcome this problem would be IMO to introduce a new option to the NavigationAgent2D so that it uses non-optimized paths for movement. In the case of tilemap the paths would go through tile centers like this (red line):

@nathanielhudson would such an option be satisfactory to you?

I think the milestone for this one should be 3.5.

This sounds like a large thing to tackle for 3.5, so I'd prefer keeping the milestone to 3.x to not disappoint people in case this needs to be pushed back.

I think the milestone for this one should be 3.5.

This sounds like a difficult problem to tackle for 3.5, so I'd prefer keeping the milestone to 3.x to not disappoint people in case this needs to be pushed back.

ok, sounds good

@Scony - Tile centers would be good for my use case, but it might be a problem for others - tile centers assumes that the entities are approximately the same size as the agents - If I had small tiles and big agents, it may not work.
I feel that ideally we'd have either the navmesh or agent be smart enough to take the agent's radius into account. Spitballing here... it would be neat if the navmesh or agent could internally cache a modified version of the navmesh geometry contracted by the agents' sizes. For example, my 10px agent goes to make a path and creates a new version of the navmesh geometry offset_polygon-ed by 10px. If that agent asks for another path the cached geometry could be re-used. If a 20px agent asked for a path, a new version with a 20px offset would be created and cached. Don't know if that's too much complexity or would have weird performance implications though.

I feel that ideally we'd have either the navmesh or agent be smart enough to take the agent's radius into account. Spitballing here... it would be neat if the navmesh or agent could internally cache a modified version of the navmesh geometry contracted by the agents' sizes. For example, my 10px agent goes to make a path and creates a new version of the navmesh geometry offset_polygon-ed by 10px. If that agent asks for another path the cached geometry could be re-used. If a 20px agent asked for a path, a new version with a 20px offset would be created and cached. Don't know if that's too much complexity or would have weird performance implications though.

IMO that's only doable with baking stage implemented.

Fair.

Fair.

It was never implemented. Till 3.5 Godot navigation system was very straightforward.

@smix8
Is there a way to use this workaround with a tilemap too, please?

This is very disappointing. I have been looking avoiding the current Godot3 Navigation2D exactly because entities get stuck on corners and was hoping the new navigation solution in Godot 3.5 and Godot4 to solve this long standing issue. It's great we now have it built-into 3D navigation, but the vast majority of Godot users are still using Godot for 2D as recent poll shows.

I have the same problem in Godot 3.5, Tile map navigation have problems because characters get stuck in corners or try to pass in 3px routes when they have a size of 100px.Also, navigation agent avoidance radius only works with obstacles, not with the tilemap navigation. I workaround the navigation by removing a chunk of navigation around the walls, but only will work if all agents have the same size and have a lot of problems. Any solution?

@AlejandroMonteseirin
The only solution is to fix your navpolygon by adding enough margin for whatever actor size should use the navigationmesh for pathfinding. The navigationmesh describes the safe-zone for the actors center position and is the baked result with all the information about collision / scene layout / actor size and not the other way around.

As a workaround, you can set the navigation region to be half the size of the tile:

Then set "Edge Connection Margin" to half the tile size. The project settings navigation/2d/default_edge_connection_margin, and also available on the Navigation node.

That is a nice workaround. If you have all entities of the same size it may work great.
If you have entities of different sizes you may still have some issues. IMO NavigationAgent2D will still need to have a radius setting so that different sized agents can use the same navmesh.

Unfortunately that work around doesn't work for my situation, as the walls in my map are much thinner than the total tile size, so if i make a margin big enough to connect the little island navmesh pieces like you have in the gif above, then the agents can form paths through the walls. I'm also quite nonplussed at how you're able to get such uniform perfect navmeshes on ur tiles, I couldn't find a grid feature when trying to make mine. there is a snap feature, but no grid:

Working on 2D NavigationMesh Baking, progress see video below.

Added parser to bake a NavigationRegion2D NavigationPolygon from TileMap / TileSet
Works the same as NavigationRegion3D works with GridMap.

@smix8 any updates on this?

@hornetDC
Will be part of pr https://github.com/godotengine/godot/pull/70724 if you want to track it. It is not part of the pr yet as I did not have time to push the update from my local working branch yet.

@smks Godot 4.1+ only.

This issue is marked as closed. Why? Is there new APIs? Because it seems it is still present in Godot 4.2.

And so, I tried to implement the workaround given here by @smix8. I used the example from the 2D Navigation Overview. I can't seem to make it work. The radius stays at 0. What am I missing?

Here is the code I'm using:
extends CharacterBody2D

var movement_speed: float = 20.0
var movement_target_position: Vector2 = Vector2(350.0, 190.0)

@onready var navigation_agent: NavigationAgent2D = $NavigationAgent2D
@onready var navigation_region: NavigationRegion2D = $"../NavigationRegion2D"

func _ready():
	# These values need to be adjusted for the actor's speed
	# and the navigation layout.
	navigation_agent.path_desired_distance = 4.0
	navigation_agent.target_desired_distance = 4.0

	# Make sure to not await during _ready.
	call_deferred("actor_setup")

func actor_setup():
	### MY ADDITION BELOW
	var navigation_map: RID = NavigationServer3D.map_create()
	var navigation_mesh: NavigationMesh = navigation_region.navigation_polygon.get_navigation_mesh()
	navigation_mesh.agent_radius = 10.0
	NavigationMeshGenerator.bake(navigation_mesh, self)
	NavigationServer3D.region_set_navigation_mesh(navigation_region.get_region_rid(), navigation_mesh)
	NavigationServer3D.region_set_map(navigation_region.get_region_rid(), navigation_map)
	NavigationServer3D.map_set_cell_size(navigation_map, 1.0)
	NavigationServer3D.map_force_update(navigation_map)
	navigation_agent.set_navigation_map(navigation_map)
	### MY ADDITION ABOVE

	# Wait for the first physics frame so the NavigationServer can sync.
	await get_tree().physics_frame

	# Now that the navigation map is no longer empty, set the movement target.
	set_movement_target(movement_target_position)

func set_movement_target(movement_target: Vector2):
	navigation_agent.target_position = movement_target

func _physics_process(_delta):
	if navigation_agent.is_navigation_finished():
		return

	var current_agent_position: Vector2 = global_position
	var next_path_position: Vector2 = navigation_agent.get_next_path_position()

	var new_velocity: Vector2 = next_path_position - current_agent_position
	new_velocity = new_velocity.normalized()
	new_velocity = new_velocity * movement_speed

	velocity = new_velocity
	move_and_slide()

I could make it work eventually. Here is the solution I used:
func actor_setup():
	### MY ADDITION BELOW
	var radius: float = -10.0
	var tab: Array[PackedVector2Array]
	var polygon = NavigationPolygon.new()
	var outline = navigation_region.navigation_polygon.get_outline(0)
	tab = Geometry2D.offset_polygon(outline, radius)
	polygon.add_outline(tab[0])
	polygon.make_polygons_from_outlines()
	navigation_region.navigation_polygon = polygon
	### MY ADDITION ABOVE

	# Wait for the first physics frame so the NavigationServer can sync.
	await get_tree().physics_frame

	# Now that the navigation map is no longer empty, set the movement target.
	set_movement_target(movement_target_position)


added docs on instrumentation libraries for swift

(Edit @chalin)
Preview: https://deploy-preview-1924--opentelemetry.netlify.app/docs/instrumentation/swift

@bryce-b I think these look pretty good. Would you like to convert from a draft and have a proper review, or are you going to add more content?

Thank you @bryce-b for your work here! It's highly appreciated.


[DPTOOLS-2735] Add method to load yaml file for connections

https://jira.lyft.net/browse/DPTOOLS-2735
Adding CLI to accept connection file and check and update the connection.
Test:
Run from devbox -- first time:

Run from devbox -- second time:

👀 @youngyjd
cc: @feng-tao

Codecov Report

Merging #187 into airflowinfra will decrease coverage by 53.38%.
The diff coverage is 0.00%.

@@                Coverage Diff                @@
##           airflowinfra     #187       +/-   ##
=================================================
- Coverage         81.19%   27.80%   -53.39%
=================================================
  Files               529      528        -1
  Lines             36598    36613       +15
=================================================
- Hits              29715    10181    -19534
- Misses             6883    26432    +19549

Impacted Files
Coverage Δ

airflow/bin/cli.py
16.12% <0.00%> (-50.46%)
:arrow_down:

airflow/www/forms.py
0.00% <0.00%> (-100.00%)
:arrow_down:

airflow/www_rbac/forms.py
0.00% <0.00%> (-100.00%)
:arrow_down:

airflow/www_rbac/widgets.py
0.00% <0.00%> (-100.00%)
:arrow_down:

airflow/www/gunicorn_config.py
0.00% <0.00%> (-100.00%)
:arrow_down:

airflow/www_rbac/blueprints.py
0.00% <0.00%> (-100.00%)
:arrow_down:

airflow/operators/mysql_to_hive.py
0.00% <0.00%> (-100.00%)
:arrow_down:

airflow/operators/mysql_operator.py
0.00% <0.00%> (-100.00%)
:arrow_down:

airflow/operators/sqlite_operator.py
0.00% <0.00%> (-100.00%)
:arrow_down:

airflow/operators/generic_transfer.py
0.00% <0.00%> (-100.00%)
:arrow_down:

... and 419 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 275b595...507177f. Read the comment docs.

👍


Allow selecting / checking values in checkboxes, radios and select by passing current value(s)

Related component
Checkboxes, radios and select
Context
We should consider allowing the user to pass the current value (or array of values, for checkboxes?), and using that to work out which item should be checked / selected.
For example, by introducing a new top level value param:
{{ govukSelect({
    id: "sort",
    name: "sort",
    label: {
    text: "Sort by"
    },
    items: [
        {
            value: "published",
            text: "Recently published"
        },
        {
            value: "updated",
            text: "Recently updated"
        },
        {
            value: "views",
            text: "Most views"
        },
        {
            value: "comments",
            text: "Most comments"
        }
    ],
    value: data['sort']
}) }}

Alternatives
At the minute the only way to pre-select or check specific items when using the Nunjucks macros for these components is to pass a boolean flag for each item, which then maps directly to the checked / selected HTML attribute for the <input> / <option>.
This means the user has to do the work to map the previous form data back to a boolean, resulting in code that might look something like this:
{{ govukSelect({
    id: "sort",
    name: "sort",
    label: {
        text: "Sort by"
    },
    items: [
        {
            value: "published",
            text: "Recently published",
            selected: (data['sort'] == "published")
        },
        {
            value: "updated",
            text: "Recently updated",
            selected: (data['sort'] == "updated")
        },
        {
            value: "views",
            text: "Most views",
            selected: (data['sort'] == "views")
        },
        {
            value: "comments",
            text: "Most comments",
            selected: (data['sort'] == "comments")
        }
    ]
}) }}

This is prone to introducing errors (especially when prototyping) – for example, any changes to the name of the field or the value of the item also need corresponding changes made to the boolean logic, potentially in multiple places.
Additional information (if applicable)
We should consider providing this in addition to the existing boolean attribute – if so, we'll need to be clear how the two interact with each other.
We should consider how this interacts with the checked helper in the Prototype Kit.

this is nice!

what if the attribute in the root had the html name for consistency (selected for select, checked for radios/checkboxes)?
would the attribute still be valid on each item as it is now? If so, they could conflict

This would be a nice addition - what about default or defaultValue?
In the Becoming a teacher service line, we have a filter that essentially does this - originally written by @fofr, and modified by me. You can see an example of it in use here. It allows us to write much more terse UI macros, focusing on what's unique about them.
Among other things, if selected or checked is not provided on an item, then it compares the data provided with each item and marks as checked as needed.

would the attribute still be valid on each item as it is now? If so, they could conflict

If selected or checked is provided on an item it does nothing. This is nice where I want 9 out of 10 options to just compare against data, but one option needs more complex logic to determine if it should be checked.

Here are some docs on how that filter works:
https://govuk-prototype-rig.herokuapp.com/docs/using-data/form-components
Extract from that page
For example, to collect a user’s email address, you could write the following:
{{ govukInput({
  label: {
    text: "Email address"
  },
  id: "email-address",
  name: "account['email-address']"
  value: "data.account['email-address']"
}) }}

3 separate parameters are needed to store and display data. Each uses the same value, but formatted in a slightly different way:

id, which uses kebab case
name, which uses object notation
value, which also uses object notation, but proceeded by data.

Radios, checkboxes and select components also need to add a checked or selected attribute to any option that matches the stored value, and date inputs need individual values for day, month and year fields.
Decorating forms
The decorate function removes this overhead. This adds name, value, id (or idPrefix) and checked/selected attributes to a GOV.UK form component, with values based on where the data is stored.
The above example can now be rewritten like this:
{{ govukInput(decorate({
  label: {
    text: "Email address"
  }
}, "account['email-address']")) }}

Note - @fofr's version and mine are subtly different in that I pass the data it should compare, and @fofr passes a string reference to the location of the data.
Using a string reference is how the checked() function works, but I find this really limiting as it forces you to only compare to exiting data in session data, rather than something you might have computed in the view. If the reference is deeply nested or you need to compute the location in the view, it can get really repetitive - you can't assign that data to a variable and pass the variable in.


Need to "Pass the argument trust_remote_code=True "

ValueError: The repository for 'internlm-chat-7b-v1_1' contains custom code which must be executed to correctlyload the model. You can inspect the repository content at https://hf.co/..../internlm-chat-7b-v1_1. Please pass the argument trust_remote_code=True to allow custom code to be run.

thanks


certbot.errors.AuthorizationError: Some challenges have failed.

I know this is boring etc.. but I honestly don't know how to fix this.
When creating a certificate for a website, It just dosen't work. The logs are as follows:
{
  "identifier": {
    "type": "dns",
    "value": "paillat.is-a.dev"
  },
  "status": "invalid",
  "expires": "2024-01-17T13:01:05Z",
  "challenges": [
    {
      "type": "http-01",
      "status": "invalid",
      "error": {
        "type": "urn:ietf:params:acme:error:connection",
        "detail": "2001:1600:13:101::80: Fetching http://paillat.is-a.dev/.well-known/acme-challenge/5O9bZP-[REDACTED]: Timeout after connect (your server may be slow or overloaded)",
        "status": 400
      },
      "url": "https://acme-v02.api.letsencrypt.org/acme/chall-v3/[REDACTED]/[REDACTED]",
      "token": "[REDACTED]",
      "validationRecord": [
        {
          "url": "http://paillat.is-a.dev/.well-known/acme-challenge/[REDACTED]",
          "hostname": "paillat.is-a.dev",
          "port": "80",
          "addressesResolved": [
            "<IP_ADDRESS>",
            "2001:1600:13:101::80"
          ],
          "addressUsed": "2001:1600:13:101::80"
        }
      ],
      "validated": "2024-01-10T13:01:05Z"
    }
  ]
}
2024-01-10 14:02:35,864:DEBUG:acme.client:Storing nonce: [REDACTED]
2024-01-10 14:02:35,865:INFO:certbot._internal.auth_handler:Challenge failed for domain paillat.is-a.dev
2024-01-10 14:02:35,865:INFO:certbot._internal.auth_handler:http-01 challenge for paillat.is-a.dev
2024-01-10 14:02:35,865:DEBUG:certbot._internal.display.obj:Notifying user:
Certbot failed to authenticate some domains (authenticator: webroot). The Certificate Authority reported these problems:
  Domain: paillat.is-a.dev
  Type:   connection
  Detail: 2001:1600:13:101::80: Fetching http://paillat.is-a.dev/.well-known/acme-challenge/[REDACTED]: Timeout after connect (your server may be slow or overloaded)

Hint: The Certificate Authority failed to download the temporary challenge files created by Certbot. Ensure that the listed domains serve their content from the provided --webroot-path/-w and that files created there can be downloaded from the internet.

2024-01-10 14:02:35,866:DEBUG:certbot._internal.error_handler:Encountered exception:
Traceback (most recent call last):
  File "/usr/local/certbot/lib/python3.12/site-packages/certbot/_internal/auth_handler.py", line 108, in handle_authorizations
    self._poll_authorizations(authzrs, max_retries, max_time_mins, best_effort)
  File "/usr/local/certbot/lib/python3.12/site-packages/certbot/_internal/auth_handler.py", line 212, in _poll_authorizations
    raise errors.AuthorizationError('Some challenges have failed.')
certbot.errors.AuthorizationError: Some challenges have failed.

2024-01-10 14:02:35,867:DEBUG:certbot._internal.error_handler:Calling registered functions
2024-01-10 14:02:35,867:INFO:certbot._internal.auth_handler:Cleaning up challenges
2024-01-10 14:02:35,867:DEBUG:certbot._internal.plugins.webroot:Removing /tmp/acme-challenge/.well-known/acme-challenge/[REDACTED]
2024-01-10 14:02:35,867:DEBUG:certbot._internal.plugins.webroot:All challenges cleaned up
2024-01-10 14:02:35,867:DEBUG:certbot._internal.log:Exiting abnormally:

I don't see any reason, apart from the fact that the /.well-known path is not being correctly served, but it should with the include of the acme-challenge.conf.
I am running the latest docker image, with crowdsec enabled.
Any help would be much appreciated

please run: docker exec npmplus echo "abc" | tee /tmp/acme-challenge/abc
and then curl skL http://paillat.is-a.dev/.well-known/acme-challenge/abc

Got an error
curl skL http://paillat.is-a.dev/.well-known/acme-challenge/abc
curl: (6) Could not resolve host: skL

I've edited my comment please retry

It returns
http://paillat.dev

Wich is the website that is set in the redirect host is set to point to.

did you tried the server reachability check?

did you tried the server reachability check?

Yes I did and It succeded.

It used to work sometimes only but not always, I would say 1 out of 5 times the challenge would work but now it stopped working completely, and I started usng dns challenge, but this time I can't since I am not the one in control of the dns zone.

https://github.com/is-a-dev/register/pull/11053 I will try to reproduce

do you have ipv6 correctly configured on your system and in your nameservers?

do you have ipv6 correctly configured on your system and in your nameservers?

I did make sure to set it up, but I will doublecheck again.

I ask because:
zoey@mx:~$ curl http://paillat.dev -6I
curl: (7) Failed to connect to paillat.dev port 80 after 311 ms: Couldn't connect to server
zoey@mx:~$ curl http://paillat.dev -4I
HTTP/1.1 301 Moved Permanently
Connection: keep-alive
Content-Length: 162
Server: GitHub.com
Content-Type: text/html
Location: https://paillat.dev/
X-GitHub-Request-Id: CC8E:2E18B3:5D81A14:5EE5B3A:65A1A644
Accept-Ranges: bytes
Date: Fri, 12 Jan 2024 20:51:58 GMT
Via: 1.1 varnish
Age: 41
X-Served-By: cache-fra-eddf8230068-FRA
X-Cache: HIT
X-Cache-Hits: 1
X-Timer: S1705092719.926596,VS0,VE2
Vary: Accept-Encoding
X-Fastly-Request-ID: c13bf9e9bc487c23e5e7c14ac6f5484c151b3ecb

I ask because:
zoey@mx:~$ curl http://paillat.dev -6I
curl: (7) Failed to connect to paillat.dev port 80 after 311 ms: Couldn't connect to server
zoey@mx:~$ curl http://paillat.dev -4I
HTTP/1.1 301 Moved Permanently
Connection: keep-alive
Content-Length: 162
Server: GitHub.com
Content-Type: text/html
Location: https://paillat.dev/
X-GitHub-Request-Id: CC8E:2E18B3:5D81A14:5EE5B3A:65A1A644
Accept-Ranges: bytes
Date: Fri, 12 Jan 2024 20:51:58 GMT
Via: 1.1 varnish
Age: 41
X-Served-By: cache-fra-eddf8230068-FRA
X-Cache: HIT
X-Cache-Hits: 1
X-Timer: S1705092719.926596,VS0,VE2
Vary: Accept-Encoding
X-Fastly-Request-ID: c13bf9e9bc487c23e5e7c14ac6f5484c151b3ecb

That is hosted on github pages currently, I did not move it yet, and what I had in npmplus was a redirect to it, wich worked without ssl.

And yes, ipv6 was indeed set up.

can you maybe start npmplus again? I want to try to curl it from outside and test ipv6

can you maybe start npmplus again? I want to try to curl it from outside and test ipv6

Yup, it just crashed because of some other error and I saw it restarting myself.

zoey@mx:~$ curl [2001:1600:13:101::80]
curl: (7) Failed to connect to 2001:1600:13:101::80 port 80 after 9 ms: Couldn't connect to server
zoey@mx:~$ curl <IP_ADDRESS>

<title>301 Moved Permanently</title>

301 Moved Permanently

NPMplus

seems to be wrong configured ipv6

https://github.com/moby/moby/issues/24379
Honestly I'll just disable ipv6 completely and everywhere at this point and see. I don't wanna spend moore time trying to fix this. I'll let you know if it fixes the issue (I have a feeling it will)

does it work now? (do you use network mode host?)

does it work now? (do you use network mode host?)

I just disabled iv6 in my dns. I didn't have time to mess witht his more and it works with ipv4, the issue being that swarm does not support ipv6 on the ingress network. The solution I came up with and will probably setup when I have time is have some service on the host forward ipv6 trafic to localhost to npmplus's port.

So yes it works


[Help] Is it possible to compile together with files?

So I am new to nuitka and wanted to compile my code
Im just going to write a basic example that relates to my code
from PIL import Image

file = "test.png"
img = Image.open(file)
img.show()

Is it possible to compile the code together with "test.png"? I want it to be literally 1 file when compiled all together with the files needed.

There is onefile mode, where you can include data files. Refer to the user manual, also on how to use sys.argv[0] and __file__ to locate them. That uses a temporary folder to run stuff. With Nuitka commercial, the files can be embedded directly. See https://nuitka.net/doc/commercial.html and https://nuitka.net/doc/commercial/protect-data-files.html for more information.


add_common_scripts-dependencies

Modified from EU with support for Facebook data.

Is the plan here to create 3 new repos for the SSF projects or a new SSF3 (is this 3?) that can serve all of them through configuration?

Is the plan here to create 3 new repos for the SSF projects or a new SSF3 (is this 3?) that can serve all of them through configuration?

3 new repos to allow flexibility in analysis requirements.


Confusion in variable names

Hello,
I have started learning about the deep learning, I have downloaded this toolbox.
To understand about the CNN I have started with the 'test_example_CNN' I am confused about these variables
train_x, test_x, train_y, test_y
why it is with 'x' and 'y'
Also what is  cnnsetup?

Those are the very basic things my friend. You need to go through the understanding the mnist dataset.
You can find the dataset under 'data' folder. The dataset is divided into training and testing sets.
Training sets have images and labels, testing sets have images only for which a label is predicted.
Hope this helps. If you are comfortable with python, You can find a python translation of the code here.
https://github.com/wajihullahbaig/DeepNeuralNetworks

Thanks Wajihullah baig,  Yes, you are right it was the basic query.
i am more comfortable with MATLAB.
some other things which I want to know is:

struct('type', 'c', 'outputmaps', 6, 'kernelsize', 5) %convolution layer
I have confusion in understanding what is meant by 'outputmaps'? 'kernelsize' 5 mean the filter size is 5x5.

struct('type', 's', 'scale', 2) %sub sampling layer
This sub sampling layer with the size 2x2? is it possible to add ReLu layer?

In the test_example_CNN if i add more convolution and sub sampling layer I got an error. what is the reason?

cnn.layers = {
    struct('type', 'i') %input layer
    struct('type', 'c', 'outputmaps', 6, 'kernelsize', 5) %convolution layer
    struct('type', 's', 'scale', 2) %sub sampling layer
    struct('type', 'c', 'outputmaps', 12, 'kernelsize', 5) %convolution layer
    struct('type', 's', 'scale', 2) %sub sampling layer
    struct('type', 'c', 'outputmaps', 18, 'kernelsize', 5) %convolution layer
    struct('type', 's', 'scale', 2) %subsampling layer
};

Error:
Array dimensions must match for binary array op.
Error in cnnbp (line 32)
z = z + convn(net.layers{l + 1}.d{j}, rot180(net.layers{l + 1}.k{i}{j}), 'full');
Error in cnntrain (line 17)
net = cnnbp(net, batch_y);
Error in test_example_CNN (line 31)
cnn = cnntrain(cnn, train_x, train_y, opts);

You should first understand the concepts of  constitutional networks. Its too lengthy to explain here. Once you grasp the ideas, you will note that it is easy to follow code or write your own. I suggest you read  about them from link/blogs/videos by andrej karpathy.

Thanks I will check that.. Is it too lengthy to give answer of each question?

You can add any sorts of layers - Just make sure you dont break the code. A ReLU is a good idea.
I am note sure why you are getting array dimension errors, possibly your data that is being convolved with a filter is some how mismatched in terms of dimensions. You should check if convn() function parameters accepted.


sorting

i know the stated intent is for a drop-in replacement of Scripting.dictionary.
but i think there would be value in adding some "bonus features"
like sorted outputs..
ie. if there was a Dict.sortedKeys option i'm sure it would get used.

that may not be a goodexample.. but being able to go
for x in Dict.sorted
    'do stuff
next

where the values come in a sorted order would be extremely useful

Hi @Shoelace thanks for the suggestion, but I'm pretty firmly in the drop-in replacement camp and have some future work planned that strictly requires it (deciding at build time whether to include this library or a Scripting Runtime reference).
Additionally, I have had some trouble finding a straightforward, cross-platform sort method so if you come up with a reusable method, please feel free to post it here and I can revisit adding it or can add it as an example.


JSON.stringify() cannot treat Java Object, from GraalJS 22.2.0

import static org.junit.Assert.assertEquals;

import org.junit.Test;

import com.oracle.truffle.js.scriptengine.GraalJSScriptEngine;

public class GraalJS {
    @Test
    public void test() throws Exception {
        org.graalvm.polyglot.Context.Builder builder = org.graalvm.polyglot.Context.newBuilder("js")
                .allowAllAccess(true);
        GraalJSScriptEngine scriptEngine = GraalJSScriptEngine.create(null, builder);

        final String script = """
                const obj = {
                    text: 'test',
                    timestamp: java.sql.Timestamp.valueOf('2022-08-13 19:57:00')
                };
                const replacer = (key, value) => Java.isJavaObject(value) ? value.toString() : value;
                JSON.stringify(obj, replacer);
                """;
        assertEquals("{\"text\":\"test\",\"timestamp\":\"2022-08-13 19:57:00.0\"}",
                     scriptEngine.eval(script));
    }
}

GraalJS 22.1.0: This unit test has been success.
GraalJS 22.2.0: The below Exception has been thrown.
javax.script.ScriptException: org.graalvm.polyglot.PolyglotException: TypeError: only JavaScript objects are supported by this operation
	at com.oracle.truffle.js.scriptengine.GraalJSScriptEngine.toScriptException(GraalJSScriptEngine.java:503)
	at com.oracle.truffle.js.scriptengine.GraalJSScriptEngine.eval(GraalJSScriptEngine.java:480)
	at com.oracle.truffle.js.scriptengine.GraalJSScriptEngine.eval(GraalJSScriptEngine.java:446)
	at java.scripting/javax.script.AbstractScriptEngine.eval(AbstractScriptEngine.java:262)
	at GraalJS.test(GraalJS.java:23)
...
Caused by: org.graalvm.polyglot.PolyglotException: TypeError: only JavaScript objects are supported by this operation
	at <js>.:program(<eval>:6)
	at org.graalvm.polyglot.Context.eval(Context.java:399)
	at com.oracle.truffle.js.scriptengine.GraalJSScriptEngine.eval(GraalJSScriptEngine.java:478)
	... 29 more

As GraalJS 22.2.0, JSON.stringify() cannot treat Java Object.

Thank you for the report. The mentioned exception is an unfortunate consequence of the fact that js.foreign-object-prototype option is set to true by default since GraalVM 22.2.0 (and the fact that our implementation of Date.prototype.toJSON does not support foreign/Java/non-JavaScript instants/dates).
We will fix the problem with Date.prototype.toJSON but you can work around this issue by setting js.foreign-object-prototype to false. This will ensure that Java Timestamp will to have Date.prototype on its prototype chain (i.e. the problematic Date.prototype.JSON will not be invoked in this case).

Thank you for your reply.
I understood the cause. And it worked to set js.foreign-object-prototype to false.
After this problem will be fixed, we will try to set js.foreign-object-prototype to true.

We will fix the problem with Date.prototype.toJSON but you can work around this issue by setting js.foreign-object-prototype to false. This will ensure that Java Timestamp will to have Date.prototype on its prototype chain (i.e. the problematic Date.prototype.JSON will not be invoked in this case).

Date.prototype.toJSON has been fixed by https://github.com/oracle/graaljs/commit/b62bd00a3320b16da48945c7643652e83e4a1ac5. Note that your unit test will not pass with this fix still because Date.prototype.toJSON (that is on the prototype chain now) will provide the stringification of the object, i.e., it will return the result of Date.prototype.toISOString(). So, the outcome will be 2022-08-13T17:57:00.000Z (not 2022-08-13 19:57:00.0).

Note that your unit test will not pass with this fix still because...

I'll be careful. Thank you.


Create schools.txt

Add file for SA Gov DfE as most public school students have an @schools.sa.edu.au email address.

@sfulham I am sorry, this is not a domain of a single school. How many schools are using this domain? Are they all high schools? Why don't they have their own email domains? In general, we don't add umbrella institutions to this repository, with exception of a few cases such as some school districts. Please provide more info about this domain.

This domain is for all public/government schools, I'm not sure if private schools get accounts with this domain.
Get Outlook for Androidhttps://aka.ms/ghei36

From: Philip Torchinsky<EMAIL_ADDRESS>Sent: Friday, February 5, 2021 9:46:35 AM
To: JetBrains/swot<EMAIL_ADDRESS>Cc: sfulham<EMAIL_ADDRESS>Mention<EMAIL_ADDRESS>Subject: Re: [JetBrains/swot] Create schools.txt (#10616)
@sfulhamhttps://github.com/sfulham I am sorry, this is not a domain of a single school. How many schools are using this domain? Are they all high schools? Why don't they have their own email domains? In general, we don't add umbrella institutions to this repository, with exception of a few cases such as some school districts. Please provide more info about this domain.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/JetBrains/swot/pull/10616#issuecomment-773666030, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AJYC47BIA3HPK5OJ2JMKOXLS5MTFHANCNFSM4W3YGLRA.

@sfulham I am sorry, this domain is for too broad scope of the schools. This repository is for email domains of particular schools and colleges only. I have to close the request without merging.
If you wish to offer free JetBrains tools to all students in your school, please send an email to<EMAIL_ADDRESS>and tell us some more details about the school.
If you need the free license for yourself or for  a few people, please apply with official document ("OFFICIAL DOCUMENT" tab at https://www.jetbrains.com/shop/eform/students).


How to test before app store uploading in xcode?

Dear Harpy Developers!!!
I had checked Harpy. it's very nice to check new version in appstore.
I had integrated Harpy in my xcode project. But I can't test update dialog in simulator.
I need to upload new version in appstore surely?
Is there any way to test before uploading in appstore?
I hope you will give me answer.
Thanks.

Hi!
Just changed the version of your app in the Info.plist or in Xcode's project settings to something that's less than the current version that's in the App Store.

App is not published in appstore yet. In this case, testing is not possible?
I had changed version 1.1. but I can't see dialog.

Ah ok, just use the App ID from a different app. The Sample App loaded in harpy uses the AppID from iTunes Connect Mobile.


Global DNS: Cannot change an entry target from a mcapp to projects

Rancher version
master 2/25
Steps to reproduce

Create a global DNS entry test.fqdn using a "gapp" multiclusterapp as the target.
Edit the entry, deselect the mcapp target.
Click resolves to Projects and choose the Default project of a cluster.
Click save

Result

There's a similar UI ticket: https://github.com/rancher/rancher/issues/17549
But I think it is also undoable from the API perspective.
We disable updating projectIds via PUT API, and require an addProject action for that purpose. When a multiclusterapp target is selected after Step 1:

If I try to do an PUT API that sets multiClusterAppId to empty, I get the following error:

If I try to call addProject action when multiClusterAppId is set:

@gitlawr @westlywright
With latest change, now UI/CLI would need to explicitly pass {"multiClusterAppId":"", "projectIds":[p1,p2]"} in the actionInput when user has unselected the multi cluster app and now wants to add projects. Earlier, we were only passing projectIds and there was no way to decide if multiClusterAppName was edited.

@kinarashah can you open a UI ticket with that info and assign it to me? @gitlawr i'll take care of the UI updates.

@kinarashah sorry no need for a UI ticket, we already have rancher/rancher#17549 that highlights the problem.

backend is in, im moving to-test, but maybe you need to wait for UI to properly test this one?

@kinarashah  I tried to test the fix by using "addProjects" option on a GlobalDNS entry that has a MulticlusterApp assocaited with it to remove the mcapp and add a project by doing the following:
HTTP Request:
HTTP/1.1 POST /v3/globalDnses/cattle-global-data:gd-2ktl4?action=addProjects
Host: santest.qa.rancher.space
Accept: application/json
Content-Type: application/json
Content-Length: 59

{
"multiClusterAppId": null,
"projectIds": [
"c-6vpvj:p-j5qnd"
]
}
HTTP Response:
HTTP/1.1 422
date: Sat, 02 Mar 2019 03:56:01 GMT
server: nginx/1.15.6
strict-transport-security: max-age=15724800; includeSubDomains
content-type: application/json
status: 422
x-api-schemas: https://santest.qa.rancher.space/v3/schemas
content-length: 184
expires: Wed 24 Feb 1982 18:42:00 GMT

{
"baseType": "error",
"code": "InvalidOption",
"message": "cannot add projects to globaldns as targets if multiclusterappID is set cattle-global-data:hello123",
"status": 422,
"type": "error"
}

Is passing "multiClusterAppId": null  OK ? Or we need to be passing "" (empty string) ?

@kinarashah @mrajashree Why are we introducing "multiClusterAppId" in addProject field ? It does not seem natural to have a "multiClusterAppId" in addProject action.  Can we instead have "addProject" action error out when the GDNS entry has a "multiClusterAppId" associated with it and provide appropriate error message to let users know that "multiClusterAppId" needs to be removed before add projects can succeed ? I understand that this will be a 2 step process for the users . But that will also ensure that the user explicitly removing  "multiClusterAppId"  before he adds a project.
@alena1108 @westlywright  ^^^

@sangeethah Talked to @mrajashree and @alena1108 Will remove the multiClusterAppId from the actions and UI will do first update an then call addProjects

Move to @westlywright for UI changes

Tested with latest build from master.
When trying to add Projects to a Global DNS entry that is already associated with MCAPP , API errors out with following exception:
{
"baseType": "error",
"code": "InvalidOption",
"message": "cannot add projects to globaldns as targets if multiclusterappID is set cattle-global-data:mcappnew",
"status": 422,
"type": "error"
}

The UI will allow a user to change to a project target or multi-cluster app target on edit.

UI aspect already tested in #17549
Refer to: https://github.com/rancher/rancher/issues/17549#issuecomment-470194479


As a user I want a some enhancements to the Create Facet feature

As a user I want a some enhancements to the Create Facet feature.
In the search tool we would like some changes to the New Facets feature to make it clear and more consistent for the user.
Acceptance criteria:

[x] Modify link text to create a new facet from 'New +' to "Add a new facet".
[x] Change text in the pop-up from "Create new aggregation" to "Create a new facet".
[x] The new created facet should show on top of the default facets.

Note: This issue is part of #262

@jblom @lilimelgar
Issue for labo-components project done in branch:
issue/#309-enhancements-create-facet-feature
What's been done?

Modify link text to create a new facet from 'New +' to "Add a new facet".
Change text in the pop-up from "Create new aggregation" to "Create a new facet".
The new created facet should show on top of the default facets.

Screencast demo
https://drive.google.com/open?id=1rVg2WxQJfqAdRoWfxLpxrOK_tvRSgrqC

@avlisodraude merged this into master after quite heavy refactoring of this component's code:

arrow functions for everything
got rid of that toggle button (exclude facets); I hated the implementation (also triggering React warnings). Replaced that with a regular checkbox
fixed a small bug. The top facet box could not be hidden.
updated several CSS class names using the IDUtil (to make sure they are unique)
Reduced 4 sorting buttons to 2 (now they alternate icon based on the sort direction)
Now you can also see which sorting option is active (different color)

@lilimelgar should check this as well.

@avlisodraude I tested in production and it all works OK. Only two things before closing:

[ ] one minor thing, for you to check  if it is a bug please: When the added facet has not items, the facet goes to the bottom, not to the top (example to reproduce: Eye Films: select URL)
[ ] Johannes and Mari? suggested that the text "Hide facet" should be "Remove facet"

@avlisodraude did you see the last remark? Could you check this one please?

@lilimelgar @jblom
Changes suggested by @lilimelgar in her last comment from Apr 10, 2019 have been implemented in labo-components branch:
issue-309-display-empty-aggr-on-top
Pull request has been requested to @jblom
https://github.com/beeldengeluid/labo-components/pull/392
Done:

Empty Facet blocks are displayed on top of aggregation blocks.
Rename Hide to Remove as suggested in the issue.

Screenshots


[Documentation Issue] Querying data (synchronous) code example is wrong

Page Name: bigquery-usage
Release: 0.11.0
>>> while retry_count > 0 and not job.complete:
...     retry_count -= 1
...     time.sleep(10)
...     query.reload()  # API request

job does not exist here and query.reload() method does not exist.

Thanks @akhenakh!

Duplicates #1551.  See #1674 for resolution.


Accelerate default config: 	Not found

I get this error.
This is my notebook. I'm check-marked LoRA and disabled the prior preservation option. Then the rests are in the default setting.
https://colab.research.google.com/drive/1Kr4Fy1_wIhg_j7_rYLpDv40FLKnnDqen?usp=share_link
Thank you
No LSB modules are available.
Description:	Ubuntu 20.04.5 LTS
diffusers==0.11.1
lora-diffusion @ file:///content/lora
torchvision @ https://download.pytorch.org/whl/cu116/torchvision-0.14.1%2Bcu116-cp38-cp38-linux_x86_64.whl
transformers==4.25.1
xformers==0.0.16rc425

Copy-and-paste the text below in your GitHub issue

- `Accelerate` version: 0.15.0
- Platform: Linux-5.10.147+-x86_64-with-glibc2.29
- Python version: 3.8.10
- Numpy version: 1.21.6
- PyTorch version (GPU?): 1.13.1+cu116 (True)
- `Accelerate` default config:
	Not found
usage: accelerate <command> [<args>] launch
       [-h]
       [--config_file CONFIG_FILE]
       [--cpu]
       [--mps]
       [--multi_gpu]
       [--tpu]
       [--use_mps_device]
       [--dynamo_backend {no,eager,aot_eager,inductor,nvfuser,aot_nvfuser,aot_cudagraphs,ofi,fx2trt,onnxrt,ipex}]
       [--mixed_precision {no,fp16,bf16}]
       [--fp16]
       [--num_processes NUM_PROCESSES]
       [--num_machines NUM_MACHINES]
       [--num_cpu_threads_per_process NUM_CPU_THREADS_PER_PROCESS]
       [--use_deepspeed]
       [--use_fsdp]
       [--use_megatron_lm]
       [--gpu_ids GPU_IDS]
       [--same_network]
       [--machine_rank MACHINE_RANK]
       [--main_process_ip MAIN_PROCESS_IP]
       [--main_process_port MAIN_PROCESS_PORT]
       [--rdzv_conf RDZV_CONF]
       [--max_restarts MAX_RESTARTS]
       [--monitor_interval MONITOR_INTERVAL]
       [-m]
       [--no_python]
       [--main_training_function MAIN_TRAINING_FUNCTION]
       [--downcast_bf16]
       [--deepspeed_config_file DEEPSPEED_CONFIG_FILE]
       [--zero_stage ZERO_STAGE]
       [--offload_optimizer_device OFFLOAD_OPTIMIZER_DEVICE]
       [--offload_param_device OFFLOAD_PARAM_DEVICE]
       [--gradient_accumulation_steps GRADIENT_ACCUMULATION_STEPS]
       [--gradient_clipping GRADIENT_CLIPPING]
       [--zero3_init_flag ZERO3_INIT_FLAG]
       [--zero3_save_16bit_model ZERO3_SAVE_16BIT_MODEL]
       [--deepspeed_hostfile DEEPSPEED_HOSTFILE]
       [--deepspeed_exclusion_filter DEEPSPEED_EXCLUSION_FILTER]
       [--deepspeed_inclusion_filter DEEPSPEED_INCLUSION_FILTER]
       [--deepspeed_multinode_launcher DEEPSPEED_MULTINODE_LAUNCHER]
       [--fsdp_offload_params FSDP_OFFLOAD_PARAMS]
       [--fsdp_min_num_params FSDP_MIN_NUM_PARAMS]
       [--fsdp_sharding_strategy FSDP_SHARDING_STRATEGY]
       [--fsdp_auto_wrap_policy FSDP_AUTO_WRAP_POLICY]
       [--fsdp_transformer_layer_cls_to_wrap FSDP_TRANSFORMER_LAYER_CLS_TO_WRAP]
       [--fsdp_backward_prefetch_policy FSDP_BACKWARD_PREFETCH_POLICY]
       [--fsdp_state_dict_type FSDP_STATE_DICT_TYPE]
       [--megatron_lm_tp_degree MEGATRON_LM_TP_DEGREE]
       [--megatron_lm_pp_degree MEGATRON_LM_PP_DEGREE]
       [--megatron_lm_num_micro_batches MEGATRON_LM_NUM_MICRO_BATCHES]
       [--megatron_lm_sequence_parallelism MEGATRON_LM_SEQUENCE_PARALLELISM]
       [--megatron_lm_recompute_activations MEGATRON_LM_RECOMPUTE_ACTIVATIONS]
       [--megatron_lm_use_distributed_optimizer MEGATRON_LM_USE_DISTRIBUTED_OPTIMIZER]
       [--megatron_lm_gradient_clipping MEGATRON_LM_GRADIENT_CLIPPING]
       [--aws_access_key_id AWS_ACCESS_KEY_ID]
       [--aws_secret_access_key AWS_SECRET_ACCESS_KEY]
       [--debug]
       training_script
       ...
accelerate <command> [<args>] launch: error: argument --mixed_precision: invalid choice: '' (choose from 'no', 'fp16', 'bf16')

@treksis Sometimes this error happens when there is a missing argument. Did you run all the cells in sequence?

Solved. By default, CONDITIONING_DROPOUT_PROB is set to 0 that was why.
Regarding to CONDITIONING_DROPOUT_PROB. It says "Probability that image (applies to both instance and class images) will be selected for dropout" Is this similar to when we train CNN, we purposely dropout to prevent overfitting? Like the higher number like 0.8 will prevent overfit because it drops more and then the lower number like 0.2, the vice versa?
thank you

thank you for the kindly answer. case closed


fetchzip: don't error out if the directory inside the archive starts with a .

Description of changes
Include hidden directories/files for the count of directories/files.
Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
22.11 Release Notes (or backporting 22.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[x] Fits CONTRIBUTING.md.

sorry, i thought it would cause a mass rebuild because a lot of things use fetchzip, but remembered that fetchurl/zip calls are content addressed. Is there a way to unrequest review?


Docs make it difficult to determine Refresh Token lifetime for SAML-federated B2B guests

https://learn.microsoft.com/en-us/azure/active-directory/external-identities/direct-federation#end-user-experience says that:

With SAML/WS-Fed IdP federation, guest users sign into your Azure AD tenant using their own organizational account. When they are accessing shared resources and are prompted for sign-in, users are redirected to their IdP. After successful sign-in, users are returned to Azure AD to access resources. Their refresh tokens are valid for 12 hours, the default length for passthrough refresh token in Azure AD.

"Default" implies that Refresh Token lifetime can be changed. However, default length for passthrough refresh token says that:

After the retirement of refresh and session token configuration on January 30, 2021, Azure AD will only honor the default values described below.

Perhaps the direct federation documentation should be clear about the fact that 12 hour Refresh Token lifetime for SAML-federated guests is set in stone if that is indeed the case.
Also, if we're going to link text that says "default length for passthrough refresh token" to another page, would it not make sense for the target document to mention the word "passthrough" at least once? I think I understand what that term means after reading a lot of the Identity Platform documentation, but it did confuse me for a while. It would be helpful to define it and then include it in various tables (for example https://learn.microsoft.com/en-us/azure/active-directory/develop/active-directory-configurable-token-lifetimes#refresh-and-session-token-lifetime-policy-properties) since it has different behaviour.

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 7b1d9428-2dbb-c80b-c2cd-b4841d3d8bd3
Version Independent ID: 20af5f8d-479c-66ec-aacd-d3f9cb79c814
Content: Federation with a SAML/WS-Fed identity provider (IdP) for B2B - Azure AD - Microsoft Entra
Content Source: articles/active-directory/external-identities/direct-federation.md
Service: active-directory
Sub-service: b2b
GitHub Login: @msmimart
Microsoft Alias: mimart

@hod-harush
Thanks for your feedback! We will investigate and update as appropriate.

@cgontario
Thanks for bringing this to our attention.
I'm going to assign this to the document author so they can take a look at it accordingly.

@msmimart
Can you please check and add your comments on this doc update request as applicable.

@cgontario - you're right, thank you for bringing this to our attention. This clarification will be made in the doc:
If the Azure AD session expires or becomes invalid and the federated IdP has SSO enabled, the user experiences SSO. If the federated user's session is valid, the user isn't prompted to sign-in again. Otherwise, the user is redirected to their IdP for sign-in.
You should see the change later today. Thanks again!


feat: support tenant

What type of PR is this?

/kind feature
What this PR does / why we need it:
this is a huge commit, include #303、#306、#326、#341、#354, and #341 are manual merged.
Which issue(s) this PR fixes:

Fixes #
Special notes for reviewers:

Does this PR introduced a user-facing change?

None

Additional documentation, usage docs, etc.:

Codecov Report

Merging #363 (cdc902d) into master (7ce88bf) will decrease coverage by 0.60%.
The diff coverage is 2.73%.

@@            Coverage Diff             @@
##           master     #363      +/-   ##
==========================================
- Coverage   12.86%   12.26%   -0.61%
==========================================
  Files         102      102
  Lines       15738    16734     +996
==========================================
+ Hits         2025     2052      +27
- Misses      13471    14439     +968
- Partials      242      243       +1

Impacted Files
Coverage Δ

pkg/apis/core/v1/registry.go
0.00% <0.00%> (ø)

pkg/models/iam/utils.go
0.00% <0.00%> (ø)

pkg/query/query.go
10.73% <0.00%> (-0.86%)
:arrow_down:

pkg/apis/core/v1/handler.go
0.30% <0.57%> (-0.01%)
:arrow_down:

pkg/models/iam/iam.go
7.76% <0.73%> (-2.29%)
:arrow_down:

pkg/authorization/rbac/rbac.go
53.44% <2.22%> (-17.87%)
:arrow_down:

pkg/server/request/request.go
80.00% <92.00%> (+5.00%)
:arrow_up:

pkg/scheme/scheme.go
100.00% <100.00%> (ø)

... and 1 more

/cc @x893675

/lgtm
/approve


The open CLI option can fail silently

Description
I'm creating a downstream application that uses the CLI. However, the open option fails most of the time. It seems likely that the cause of failure is on my machine, but the failure is difficult to debug because no error or warning is displayed. If I'm understanding the code correctly, it seems like potential errors are ignored on this line. Could it instead display a warning message?
Reproduction URL
No response
Operating system
Linux
Typst version

[X] I am using the latest version of Typst

Shouldn’t open_file call open::{with_detached, that_detached} instead of open::{with_in_background, that_in_background} to have the open process live even after Typst has exited?

Shouldn’t open_file call open::{with_detached, that_detached} instead of open::{with_in_background, that_in_background} to have the open process live even after Typst has exited?

Not sure if it is intended that the pdf viewer should be a subprocess of typst and terminate when typst exits. If so, current behavior is wrong, no matter the --open flags is processed successfully or not.

I tested the --open flags locally (on linux):

typst compile /path/to/typst/file --open: opens the viewer (okular) and typst exits, and the okular process is adopted by systemd --user
typst compile /path/to/typst/file --open $(which okular): same as No.1
typst compile /path/to/typst/file --open /path/nonexistent: exit-code is 0, but no viewer opened, as described in this issue
typst compile /path/to/typst/file --open $(which git): same as No.3

Prior art: https://github.com/typst/typst/pull/706

Not sure if it is intended that the pdf viewer should be a subprocess of typst and terminate when typst exits.

@Coekjan Given that my failures were intermittent and your PR seems to have fixed them, would that mean that my machine is failing to hand off the default viewer process when Typst exits? This isn't a very important question, given you fixed it, but I am curious and don't know much about processes :)

Would that mean that my machine is failing to hand off the default viewer process when Typst exits?

@jakobjpeters Maybe you can first investigate if the viewer process is created, through tools like pstree, top (or htop), etc.


Replace DummyLayout with mocking object and improve test design

Fixes CONFIGURATION-810
Description
Refactor test class DummyLayout and test file TestPropertiesConfiguration.java.

Motivation

Decouple test class DummyLayout from production class PropertiesConfigurationLayout.
Make test logic more clear by removing the overridden method in DummyLayout.
Make test condition more explict by directly using Mockito.verify() to verify method execution status.

Key changed/added classes in this PR

Add Mockito as testing dependency.
Created mocking object to replace test subclass DummyLayout, decoupled test from production code.
Remove the int variable that used to keep tracking the invocation times of load(PropertiesConfiguration, Reader).
Use Mockito.verify() to check execution status of load(PropertiesConfiguration, Reader).

Thank you for the PR! I've left some comments on the Jira issue explaining my decision to not include this.


Ensure no null values are used when add/set headers.

Motivation:
We need to ensure we never allow to have null values set on headers, otherwise we will see a NPE during encoding them.
Modifications:
Add null check.
Result:
Correctly throw exception when a null header value is added/set

@trustin please check

LGTM

Cherry-picked into 4.0 (de1f1a61f3ddbbe2e8f706f24bab05487ab4d88f) and cherry-picked unit-test (as the issue not exists in 4.1 and master) to 4.1 (e9a2cac16df7cd6911b7032deee8bdb3ec3e3022) and master (82e580dc7f825bbf57d86a75c6a04ea03ea5b4ab)


Bump planout.js major version to v4.0.0

This PR consists of changes from the following PR's:

https://github.com/HubSpot/PlanOut.js/pull/59
https://github.com/HubSpot/PlanOut.js/pull/60

V4.0 Changes:

Core compatible bundle fixes:

Core compatible namespace allocations now match core reference namespace allocations from python version of planout.
Core compatible interpreted experiment enrollment now matches core reference interpreted experiment enrollment from python version of planout.

Separated the concerns of planout core random operations, and the planout API. The planout.js API (experiment, assignment, namespace, etc) are now composed with the random operations they are passed.
Added planoutAPIFactory.js to keep planout bundles consistent.
Made experiment & namespace names required to fix https://github.com/HubSpot/PlanOut.js/issues/57

@rawls238 @geoffdaigle @jcwilson @eytan

It occurs to me that Experiment.getDefaultParamNames() will suffer from the same problems as the removed getDefaultNamespaceName() and getDefaultExperimentName() in #57.  It tries to examine the assign() function's code and does a lexical search for set( in order to extract param names. The set() function will probably get munged during minification.

@jcwilson I think we'll be safe there since things like object properties / methods shouldn't be munged in a minification process (doing so would break any runtime code that depends on specific property / method names). Though I do want to find a way to make the implementation of getDefaultParamNames() less magical - https://github.com/HubSpot/PlanOut.js/issues/56

I guess the question is whether or not we hold up the v4 release over it. I'd be ok leaving it as-is for now.

I think we'll be okay to move forward with the v4 release before tackling that issue. The toString() matching is gross, but I don't think it's likely to affect a lot of people. And if it does affect people, then it will only hinder autoexposure logging (enrollment & assignment would be unaffected). For context, I left a followup comment in https://github.com/HubSpot/PlanOut.js/issues/56 explaining how we ran into this problem.

Cool, sounds good.
And yeah, that example you posted in #56 was exactly why I called it a "trap" :)

Yes it is in fact immune to minification

@mattrheault is this ready for review or are you planning on adding more to this?

copying this in here since I wanted to respond to this but it seems to have been lost since it was an inline-comment
Unfortunately I think we're wrestling a bit with the current structure of the project to force it to work the way that we want. In an ideal world, we shouldn't have to apply any "overrides" to our random operations at runtime.

The direction that I think PlanOut.js should move in is one where we totally separate the concerns of API & random operations. PlanOut.js's experiments, namespaces, interpreter, etc should be capable of being composed from the top with whatever random operations we choose. Right now we're in an awkward middle ground where some stuff is composable, but other stuff directly imports random operations.

Here's a peak into what I'm imagining:

/
/build
index.js
index_core_compatible.js
/es6
/api
...
/ops
...
I'm not married to this exact directory structure, but the important detail is that we should have a hard rule that no module in /api will ever import directly from /ops. Anything that an api module needs from /ops should be passed down from the build's entry point.

With this restructuring, I think that PlanOut.js will be in a good place to scale without having to worry about future changes causing a shift in enrollment for either the compat or non-compat builds.

Taking a step back to the problem at hand...

The fact is that things are broken, and they need to be fixed. The question I'm interested in answering right now is - do we apply a bandaid to fix core-compat, or do we prioritize a more involved project restructure?

@jcwilson do you have any thoughts on the matter? How high priority is fixing core-compat to you and your team?

@rawls238 do you think that what I'm proposing sounds like a reasonable direction to head in?

/cc @geoffdaigle
```quote

I think it's best to not overcomplicate things with `core-compat`.

First, I don't think that many people (if any) require compatibility and the non-compat version is *substantially* faster than the compat version. I'd be interested to hear about @jcwilson's use-case with respect to this.

Second, I don't believe it would be desirable to untangle random operations from everything since namespaces directly rely on random ops to do namespace enrollment (as they should) and the assignment class executes the random operators so I don't see how this would be a significant improvement. It's not clear to me how it would be possible to divorce the two - if there are any issues with randomization they necessarily will always impact enrollment based on the current implementation and it isn't clear to me how changing around directory structure or how random ops are injected into the build would make any difference (I have a feeling I'm missing something here, so if I am let me know).

I think that @jcwilson's changes are sufficient for fixing the issue at hand and present a good framework for moving forward and I don't think a project overhaul is necessary. It is also worth pointing out that these kinds of things should be incredibly infrequent since the reference implementation rarely, if ever, will change these core aspects of the project so it seems like we would be overengineering to completely refactor the project to accommodate the slim chance that we need to deal with this issue again.

First, I don't think that many people (if any) require compatibility and the non-compat version is substantially faster than the compat version. I'd be interested to hear about @jcwilson's use-case with respect to this.

One of the core benefits of Planout is its ability for clients and servers to have consistent hashing without having to communicate the namespace/experiment data at runtime, say via an api call or server-side rendered template. In our case, we have also have tooling built around the Python implementation that dev and analytics can use to quickly identify if a particular user is selected for an experiment. In our case, we value that consistency over the speed benefits provided by the default distribution of PlanOut.js (though we do appreciate that effort).
My understanding was that this PR accomplishes @mattrheault's goals with regards to any refactoring, so I think we're good there.

@rawls238

First, I don't think that many people (if any) require compatibility and the non-compat version is substantially faster than the compat version. I'd be interested to hear about @jcwilson's use-case with respect to this.

I think the core compat version is more likely to be used in a node.js environment where the performance loss can be mitigated.

Second, I don't believe it would be desirable to untangle random operations from everything since namespaces directly rely on random ops to do namespace enrollment (as they should) and the assignment class executes the random operators so I don't see how this would be a significant improvement. It's not clear to me how it would be possible to divorce the two - if there are any issues with randomization they necessarily will always impact enrollment based on the current implementation and it isn't clear to me how changing around directory structure or how random ops are injected into the build would make any difference (I have a feeling I'm missing something here, so if I am let me know).

Namespace classes directly importing random operations is fine until we run into a situation where we have multiple bundles where random operations differ. Now that this is the case, things like Namespaces & Experiments should be handed the random operations they need to do their job, rather than importing random operations directly. The restructuring to make this happen isn't too heavy handed IMO, and I don't anticipate any negative side effects. The only thing this effects is the developer experience when contributing to planout.js.
We are running into real breaking problems here because of these imports. I disagree in that what we're doing here is a major overhaul. It's more of a minor refactor around where dependencies come from, which will alter enrollment, and thus will require a major version bump.
Does this clear things up?

One of the core benefits of Planout is its ability for clients and servers to have consistent hashing without having to communicate the namespace/experiment data at runtime, say via an api call or server-side rendered template. In our case, we have also have tooling built around the Python implementation that dev and analytics can use to quickly identify if a particular user is selected for an experiment. In our case, we value that consistency over the speed benefits provided by the default distribution of PlanOut.js (though we do appreciate that effort).

Cool to hear you're doing this! Theoretically this is one of the benefits of using PlanOut but in practice virtually all of the devs that I've found are using it are using a single implementation of it. Glad to see you guys are using multiple implementations!

I think the core compat version is more likely to be used in a node.js environment where the performance loss can be mitigated.

This is not recommended practice. Nobody should be using core compat mode unless they need compatibility, whether on the client or server the performance differences are substantial.

Namespace classes directly importing random operations is fine until we run into a situation where we have multiple bundles where random operations differ. Now that this is the case, things like Namespaces & Experiments should be handed the random operations they need to do their job, rather than importing random operations directly. The restructuring to make this happen isn't too heavy handed IMO, and I don't anticipate any negative side effects. The only thing this effects is the developer experience when contributing to planout.js.

We are running into real breaking problems here because of these imports. I disagree in that what we're doing here is a major overhaul. It's more of a minor refactor around where dependencies come from, which will alter enrollment, and thus will require a major version bump.
I see I see, I was envisioning that you were saying something else (which in my mind made no sense) but this makes a lot of sense. I think in the short-term we should stick to these changes as I don't think they're a band-aid fix but rather a sensible restructuring. In the long-term I'd be interested to see the implementation of something like this and see if it makes things easier on the dev side than the current implementation. The only worry I have is doing something like that and breaking things for no performance or external usability reasons but I think this project has good enough test coverage where it should be OK. Up to you if you want to attempt at doing so, I'm not sure how big the gains would actually be but they seem positive to me.
Doing this after merging this in shouldn't require a major version bump since enrollment won't be affected if we do this kind of refactoring after this fix has already been applied.

Thanks for everyone's help on this. I'm excited to start using v4.0! 💯

@jcwilson would you prefer that we put the compat planout implementation as its own package on npm?


Added support for local_file attribute in splunk_app resource

I refactored a small portion of the splunk_app LWRP and added additional functionality.  While testing, I also cleaned up a few failing tests.
Signed-off-by: Michael Kristan<EMAIL_ADDRESS>Description

Added support for local_file attribute in splunk_app resource
Added attribute for suppressing sensitive commands
Updated documentation in README.md
Fixed a majority of unit tests, mostly caused by deprecation of commands in Chef 14

Issues Resolved
Fixes issues #111 and #101
Check List

[ ] All tests pass. See https://github.com/chef-cookbooks/community_cookbook_documentation/blob/master/TESTING.MD
[X] New functionality includes testing.
[X] New functionality has been documented in the README if applicable
[X] All commits have been signed for the Developer Certificate of Origin. See https://github.com/chef-cookbooks/community_cookbook_documentation/blob/master/CONTRIBUTING.MD

Since this PR is relatively old and the last commit did not pass all tests, I'm closing the PR. If an update is warranted, please rebase with the current version and reopen or create a new PR.