pebble service cannot start with Error: open /etc/webhook/certs/cert.pem: no such file or directory

This issue is to document the investigation of the bug reported in https://github.com/canonical/bundle-kubeflow/issues/826
Description
the admission-webhook pebble service does not start, with the error:
config=main.Config{CertFile:"/etc/webhook/certs/cert.pem", KeyFile:"/etc/webhook/certs/key.pem"} Error: open /etc/webhook/certs/cert.pem: no such file or directory
2024-02-13T11:53:36.729Z [container-agent] 2024-02-13T11:53:36Z ERROR cannot start service: exited quickly with code 255

The webhook service needs to find the certificate files (cert.pem and key.pem) in the workload container directory /etc/webhook/certs.
When these cert files are not found, this puts the workload into an usable state, where the webhook service is not up. This is problematic because users cannot create Notebooks, as reported in the issue.
Case where the flow works correctly
In the charm, we upload the certificates to the container on pebble-ready event, so the event sequence for this to be successful is:

install -> leader_elected -> admission_webhook_pebble_ready -> config_changed
the admission_webhook_pebble_ready handler is:
https://github.com/canonical/admission-webhook-operator/blob/0707be37a97d2f52b785caf670ec98aa66d97ce9/src/charm.py#L305-L311
where on_event method is:
https://github.com/canonical/admission-webhook-operator/blob/0707be37a97d2f52b785caf670ec98aa66d97ce9/src/charm.py#L325-L346

and the config_changed handler is set to on_event
the key thing to observe here is when the first call of update_layer is, because that is when the service is attempted to start.
chisme's update_layer is where the Pebble service is getting started if the pebble service definition was updated, the method in chisme:
def update_layer(container_name: str, container: Container, new_layer: Layer, logger: Logger):
    """Updates the Pebble configuration layer if changed.

    Args:
        container_name (str): The name of the container to update layer.
        container (ops.model.Container): The container object to update layer.
        new_layer (ops.pebble.Layer): The layer object to be updated to the container.
        logger (logging.Logger): A logger to use for logging.
    """
    if not container.can_connect():
        raise ErrorWithStatus("Waiting for pod startup to complete", MaintenanceStatus)

    current_layer = container.get_plan()

    if current_layer.services != new_layer.services:          # HERE: this is the check referred to
        container.add_layer(container_name, new_layer, combine=True)
        try:
            logger.info("Pebble plan updated with new configuration, replanning")
            container.replan()
        except ChangeError:
            logger.error(traceback.format_exc())
            raise ErrorWithStatus("Failed to replan", BlockedStatus)

the code flow would be:

pebble_ready handler executes -> certificates are pushed to container -> on_event is called -> update_layer first call so the service is started correctly
config_changed handler executes on_event -> update_layer is called for the second time -> services equivalence check does not pass, so the service is not restarted

Cases where it can go wrong
If update_layer gets called for the first time when the cert files are not yet uploaded, this can happen when the following conditions are true:

any event that calls update_layer gets fired before pebble_ready
container.can_connect() returns True, so the update_layer function does not raise an Error and proceeds to execute container.replan() which starts the service

We know that condition 1. is possible for charm execution, for example config_changed can happen before pebble_ready
For condition 2. to apply, this means that pebble_ready event was fired as well, BUT there is no guarantee that pebble_ready handler has executed, it is a possible race condition here.

Race condition
the error-prone scenario is like this:

config_changed is fired
config_changed handler execution starts
before config_changed handler execution reaches the container.can_connect() check in update_layer, the container becomes ready
pebble_ready event is queued, but the config_changed handler is in the middle of execution so it continues
config_changed handler container.can_connect() check in update_layer returns True
config_changed handler adds the layer and calls container.replan()
the error is hit because the cert files are not found
pebble_ready handler executes
the service equivalence check if current_layer.services != new_layer.services: returns False because the service is the same
pebble_ready handler skips adding the layer and container.replan() i.e. it does not restart the service

At this state, the charm is unable to recover from the error, because every handler will skip restarting the service.

Health checks
the admission-webhook charm has a pebble health check in place:
    def _admission_webhook_layer(self) -> Layer:
        """Create and return Pebble framework layer."""
        layer_config = {
            "summary": "admission-webhook layer",
            "description": "Pebble config layer for admission-webhook-operator",
            "services": {
                self._container_name: {
                    .....
                    # Service details
                    .....
                    "on-check-failure": {"admission-webhook-up": "restart"},
                },
            },
            "checks": {
                "admission-webhook-up": {
                    "override": "replace",
                    "period": "30s",
                    "timeout": "20s",
                    "threshold": 4,
                    "tcp": {"port": self._port},
                }
            },
        }

If functioning correctly, the pebble service should be restarted when the service port is not listening on self._port
From what we see in the logs, the check does fail, but it is not restarting the service.
This is unexpected behavior from Pebble.
Issues with the pebble check

we see in the logs Check "admission-webhook-up" failure threshold 4 hit, triggering action when the threshold is hit for the first time, and yet the service is not restarted.
we see this log Check "admission-webhook-up" failure threshold 4 hit, triggering action only the first time that the threshold is hit, then all the other reported failures do not log this.

closing since this was fixed in #125 and backported in #126


NPG-3847 Add OpenTelemetry layer to vit-ss server

I believe I can do that in a separate PR as it would touch multiple crates. WDYT?


Input-field label misaligned

Label for an input within input-field is inlined after an input instead of being over an input.
Most modern browsers:

IE11:

Here is the [JSBin snippet][1], which demonstrates this wrong behavior. It works great in all browsers, except for IE.
After some investigations, it looks like the problem is that it is somehow related to position, display and absence of left CSS property.
[1]: http://jsbin.com/giroviquni/2/edit?html,output

Currently, the left property is applied to the input field label, only if input field has the .col class
https://github.com/Dogfalo/materialize/blob/master/sass/components/forms/_input-fields.scss#L129


Add switch to toggle if agents should be paused when the environment is resumed

Description
closes #2091
Self Check:
Strike through any lines that are not applicable (~~line~~) then check the box

[x] Attached issue to pull request
[x] Changelog entry
[x] Code is clear and sufficiently documented
[x] Sufficient test cases (reproduces the bug/tests the requested feature)
[x] Correct, in line with design
[x] ~End user documentation is included or an issue is created for end-user documentation (add ref to issue here: )~

Reviewer Checklist:

[ ] Sufficient test cases (reproduces the bug/tests the requested feature)
[ ] Code is clear and sufficiently documented
[ ] Correct, in line with design

Processing this pull request

An unexpected error occurred while processing this pull request. The request might have been partially processed. Caused by: Remote state does not match local state: got pull requests for branches master but change entry specifies branches master, iso5.
If this is a first stage pull request (not opened by the merge tool) this is probably caused by a race condition (inmanta/irt#650). The best way to resolve this is to make sure the merge tool can pick up this pull request again and start from a clean slate. To do this, take the following steps:

Close all pull requests opened by the merge tool with the same source branch as this one. Be careful not to close pull requests for source branches that start with merge-tool/. All pull requests created by the merge tool link to this pull request so you should be able to find them mentioned here.
Re-add the merge-tool-ready label to this pull request.

Processing this pull request

Merged into branches master, iso5 in 13adac3d165c31ab0fa7ccab218b39ee72df36ce


Issue in unity

Hi, I have this issue.

https://github.com/itisnajim/SocketIOUnity#receiving

https://github.com/itisnajim/SocketIOUnity#receiving

Can you be more detailed? I don't get it.

@rihno123, to retrieve your object, it is recommended to use the GetValue method. For example, to get an object of type 'SomeClass':
var obj = response.GetValue<SomeClass>();

If the object is a string, you can use:
string text = response.GetValue<string>();

Alternatively, if you wish to view the object as a stringified JSON, you can use:
string jsonStr = response.GetValue(0).GetRawText();

@rihno123, to retrieve your object, it is recommended to use the GetValue method. For example, to get an object of type 'SomeClass':
var obj = response.GetValue<SomeClass>();

If the object is a string, you can use:
string text = response.GetValue<string>();

Alternatively, if you wish to view the object as a stringified JSON, you can use:
string jsonStr = response.GetValue(0).GetRawText();

Ok, but when I use that line I get this error.

Hi, I have the same issue as the first screenshot, I'm at Unity version 2021.3.17f1.
I imported the package without issues, but when I copied the sample folder then the unity console log shows that error.
As I'm just testing this package for the GGJ, I replaced the error line response.GetValue().GetRawText() with response.ToString() and seems to work, but I'll really appreciated if you can update the sample directory please.


close http response body when attaching

close http response body when attaching

I'm not sure this is correct.
We're already passing resp to holdHijackedConnection, at which point the caller should not be calling close on it anymore.
Beyond that, this would need to be done from inside the promise otherwise you risk closing before the goroutine is ready to yield control of it.

@cpuguy83
After I read what you said, I am not sure, either.
But what still confuse me is that if no one close this, there must be possibility to have memory leak, and can not reuse that fd resource.
While, in both places calling ContainerAttach:
https://github.com/docker/docker/blob/master/api/client/attach.go#L71-96
https://github.com/docker/docker/blob/master/api/client/start.go#L97-112
,Close added to resource leak.

any update?

I don't think that it creates memory leak, but I don't see harm in closing as well. Goroutine must be long dead by time when this function ends.
LGTM

ping @cpuguy83 PTAL if you still have objections

I don't see any risk there (famous last words). LGTM, thanks!


wrong-type-argument window-live-p #<window 69>

What did you expect to happen?
Using org-roam - opening the org-roam buffer to check backlinks. Should be presented with a window and buffer that lists the backlinks. instead the error wrong-type-argument window-live-p #<window 69> is displayed.
Closing the org-roam buffer and opening it again does not show the error, however, the backlinks buffer (org-roam) is a blank empty buffer.
What actually happened?
with the doom-debug-mode enabled the following error is shown in place of the backlink buffer
Additional details:

Explain-pause-mode: please report this bug by creating a Github
issue at https://github.com/lastquestion/explain-pause-mode. Explain-pause-mode
is now _disabled_ so you can continue to hopefully use Emacs. Info follows:
explain-pause version: 0.1
emacs version: 27.1
not top level in wrap-native for #
current
#s(explain-pause-command-record root-emacs nil nil 25162 (24465 43147<PHONE_NUMBER>00) nil nil nil nil 0)
Backtrace:
explain-pause-report-measuring-bug("not top level in wrap-native for #<subr recursive-..." "current" #s(explain-pause-command-record :command root-emacs :native nil :parent nil :executing-time 25162 :entry-snap (24465 43147<PHONE_NUMBER>00) :too-slow nil :is-profiled nil :under-profile nil :profile nil :depth 0))
explain-pause--wrap-native(#)
apply(explain-pause--wrap-native # nil)
recursive-edit()
debug(error (wrong-type-argument window-live-p #<window 69>))
redisplay_internal\ (C\ function)()

Steps to reproduce:

Open an org-roam file
open the backlinks buffer (assumes that it does not automatically show)
error happens

System information:

SYSTEM  type       gnu/linux
	config     x86_64-pc-linux-gnu
	shell      /bin/bash
	uname      Linux 5.4.0-52-generic #57-Ubuntu SMP Thu Oct 15 10:57:00 UTC 2020 x86_64
	path       (~/.pyenv/shims ~/.pyenv/bin ~/.local/bin ~/bin /usr/local/sbin /usr/local/bin /usr/sbin /usr/bin /sbin /bin /usr/games /usr/local/games /snap/bin ~/.emacs.d/bin /snap/bin /var/lib/snapd/snap/bin /usr/lib/emacs/27.1/x86_64-linux-gnu)
EMACS   dir        ~/.emacs.d/
	version    27.1
	build      Sep 19, 2020
	buildopts  --build=x86_64-linux-gnu --prefix=/usr '--includedir=${prefix}/include' '--mandir=${prefix}/share/man' '--infodir=${prefix}/share/info' --sysconfdir=/etc --localstatedir=/var --disable-silent-rules '--libdir=${prefix}/lib/x86_64-linux-gnu' '--libexecdir=${prefix}/lib/x86_64-linux-gnu' --disable-maintainer-mode --disable-dependency-tracking --prefix=/usr --sharedstatedir=/var/lib --libexecdir=/usr/lib --localstatedir=/var/lib --infodir=/usr/share/info --mandir=/usr/share/man --enable-locallisppath=/etc/emacs:/usr/local/share/emacs/27.1/site-lisp:/usr/local/share/emacs/site-lisp:/usr/share/emacs/27.1/site-lisp:/usr/share/emacs/site-lisp --program-suffix=27 --with-modules --with-file-notification=inotify --with-mailutils --with-harfbuzz --with-json --with-x=yes --with-x-toolkit=gtk3 --with-lcms2 --with-cairo --with-xpm=yes --with-gif=yes --with-gnutls=yes --with-jpeg=yes --with-png=yes --with-tiff=yes --with-xwidgets 'CFLAGS=-g -O2 -fdebug-prefix-map=/build/emacs27-bifpWT/emacs27-27.1~1.git86d8d76aa3=. -fstack-protector-strong -Wformat -Werror=format-security -no-pie' 'CPPFLAGS=-Wdate-time -D_FORTIFY_SOURCE=2' 'LDFLAGS=-Wl,-Bsymbolic-functions -Wl,-z,relro -no-pie'
	features   XPM JPEG TIFF GIF PNG RSVG CAIRO SOUND GPM DBUS GSETTINGS GLIB NOTIFY INOTIFY ACL LIBSELINUX GNUTLS LIBXML2 FREETYPE HARFBUZZ M17N_FLT LIBOTF ZLIB TOOLKIT_SCROLL_BARS GTK3 X11 XDBE XIM MODULES THREADS XWIDGETS LIBSYSTEMD JSON PDUMPER LCMS2 GMP
	traits     (batch server-running envvar-file)
DOOM    dir        ~/.doom.d/
	version    2.0.9
	build      HEAD -> bibtex c10a09bc0 2020-10-22 09:35:31 +0400
	elc-files  141
	modules    (:completion (company +childframe) (helm +fuzzy) :ui doom doom-dashboard hl-todo hydra modeline nav-flash ophints (popup +defaults) treemacs unicode vc-gutter (window-select +numbers) workspaces zen :editor file-templates fold (format +onsave) snippets word-wrap :emacs (dired +icons) electric (ibuffer +icons) (undo +tree) vc :term eshell vterm :checkers (syntax +childframe) (spell +aspell +everywhere) grammar :tools biblio (debugger +lsp) docker editorconfig (eval +overlay) (lookup +dictionary +offline +wordnet +docsets) (lsp +peek) magit make pass pdf taskrunner :lang (cc +lsp) data emacs-lisp (go +lsp) latex markdown (org +dragndrop +pandoc +present +gnuplot +roam +noter) (python +lsp +pyenv) (sh +lsp) web :app (rss +org) :config literate (default +bindings +smartparens))
	packages   ((elfeed-goodies) (org-mode :recipe (:repo https://code.orgmode.org/bzg/org-mode.git :files (*.el lisp/*.el contrib/lisp/*.el)) :pin 8402c4a778) (org-roam-server) (company-tabnine) (super-save) (crux) (imenu-anywhere) (atomic-chrome))
	unpin      (org-roam helm)
	elpa       (n/a)

I cannot reproduce this on the pinned version of org-roam. Please repin org-roam and let me know if that fixes the issue.

sorry I had been testing combinations to see what might fix it. But the pinned and unpinned were the same after your recent bump, so I left the unpinned.
What happened to be the issue is that I ended up with a lot of cyclical links, I recently changed to using org-id and the combination of org-roam + org-id in some cases lead to these cycles. I will raise an issue with org-roam that they could notify, or maybe even identify if it can not create the backlinks because of this recursion.

I can confirm seeing the same problem. I, too, have switched to using file level org-ids... it happens in both pinned and unpinned versions of org-roam currently...
Is this a doom problem or org-roam problem?


Restore compatibility with knife-windows

knife-windows bootstrap context inherits from Chef::Knife::Core::BootstrapContext
and uses the Chef 11 arity in its constructor (https://github.com/opscode/knife-windows/blob/master/lib/chef/knife/core/windows_bootstrap_context.rb#L36)
In order to protect compatibility with knife-windows make sure that we have a default
for the new parameter that is added to the BootstrapContext constructor.
/cc: @opscode/client-engineers

Hi. I am an automated pull request bot named Curry. There are commits in this pull request whose authors are not yet authorized to contribute to Chef Software, Inc. projects or are using a non-GitHub verified email address. To become authorized to contribute, you will need to sign the Contributor License Agreement (CLA) as an individual or on behalf of your company. You can read more on Chef's blog.
Non-GitHub Verified Committers
There are 1 commit author(s) whose commits are authored by a non-GitHub verified email address. Chef will have to manually verify that they are authorized to contribute.
Please sign the CLA here.


JavaFX requirement documented

Modified the README.md file, documenting the requirement of Java library, JavaFX.

The link you've provided doesn't work
Also, JavaFX is no longer provided by Oracle.
It's Gluon who provides JavaFX.

Hello @thesherrinford , Thank you for the suggestion. I have made a new PR modifying it. Thank you. :) @jbeorse ,please review.


[module] set same version as in root

Thanks @andreyluchianic. There was already another PR with the same change, but I hadn't noticed it until I saw yours. Just in case you were wondering why your commit shows up empty, that's the reason!


Is PSNR evaluation on raw or sRGB?

In the CVPR version, all quantitative evaluations are performed on Raw Space


Logged-out home page

Should this include a collection of bookmarks?

Dependent on #96

We need documentation of what appears on the Logged out homepage, unless that already exists?

Closing this for now - we can recreate from a proper spec later after we get UX.


[#1092] Adopt proposal API

Closes #1092

TCA sdkSynchronizer dependency extended with 3 new Proposal APIs
Typical Fee < 0.001 localized and updated in the UI
Awaiting all transaction results with use of new proposal.transactionCount() method
Implemented new PartialProposalError reducer and view
Contact support mail with transaction IDs logic implemented
Fallback to share logic implemented
PPE integrated into SendFlow
PPE  integrated into Shielding
Changelog updated

This code review checklist is intended to serve as a starting point for the author and reviewer, although it may not be appropriate for all types of changes (e.g. fixing a spelling typo in documentation).  For more in-depth discussion of how we think about code review, please see Code Review Guidelines.
Author

[ ] Self-review: Did you review your own code in GitHub's web interface? Code often looks different when reviewing the diff in a browser, making it easier to spot potential bugs.
[ ] Does the code abide by the Coding Guidelines?
[ ] Automated tests: Did you add appropriate automated tests for any code changes?
[ ] Code coverage: Did you check the code coverage report for the automated tests?  While we are not looking for perfect coverage, the tool can point out potential cases that have been missed.
[ ] Documentation: Did you update Docs as appropiate? (E.g README.md, etc.)
[ ] Run the app: Did you run the app and try the changes?
[ ] Did you provide Screenshots of what the App looks like before and after your changes as part of the description of this PR? (only applicable to UI Changes)
[ ] Rebase and squash: Did you pull in the latest changes from the main branch and squash your commits before assigning a reviewer? Having your code up to date and squashed will make it easier for others to review. Use best judgement when squashing commits, as some changes (such as refactoring) might be easier to review as a separate commit.

Reviewer

[ ] Checklist review: Did you go through the code with the Code Review Guidelines checklist?
[ ] Ad hoc review: Did you perform an ad hoc review?  In addition to a first pass using the code review guidelines, do a second pass using your best judgement and experience which may identify additional questions or comments. Research shows that code review is most effective when done in multiple passes, where reviewers look for different things through each pass.
[ ] Automated tests: Did you review the automated tests?
[ ] Manual tests: Did you review the manual tests?You will find manual testing guidelines under our manual testing section
[ ] How is Code Coverage affected by this PR? We encourage you to compare coverage before and after your changes and when possible, leave it in a better place. Learn More...
[ ] Documentation: Did you review Docs, README.md, LICENSE.md, and Architecture.md as appropriate?
[ ] Run the app: Did you run the app and try the changes? While the CI server runs the app to look for build failures or crashes, humans running the app are more likely to notice unexpected log messages, UI inconsistencies, or bad output data.

For ZIP 320 support, the case where only the first transaction succeeds is essentially identical to the case where the exchange sends back the funds: both leave the funds in the ephemeral t-addr. So why can't we just handle this correctly?

For ZIP 320 support, the case where only the first transaction succeeds is essentially identical to the case where the exchange sends back the funds: both leave the funds in the ephemeral t-addr.

For ZIP 320 support, from a control of funds position, yes. But they are semantically two very different cases, and we have not done the UX work to distinguish them, or handle how the resulting transaction history is conveyed to users.

I was under the impression that handling the latter was required for Zashi 1.0.

I was not: my understanding is that we need to be able to send to TEX addresses (which covers the majority use case), but not that we need to handle exchanges sending funds back (which is an edge case).
I think we should handle both exchanges sending funds back, and transaction submission failures, in-app eventually. But I also think that handling this properly with good UX requires more engineering than we have time for in the 1.0 timeline, so it would go in a post-1.0 release. Having an error indicating to users when they have hit this edge case is the minimal amount of engineering we can do here, while avoiding those edge case users believing they have lost funds.

So why can't we handle the former correctly?

There are at least four different states we could be in here that I can think of (for ZIP 320 proposals; more if we also consider arbitrary proposals which the SDK might generate or need to handle in future):

Transient failure to submit the original transactions; logical transaction is unfinished.
Permanent failure to submit the original transactions; logical transaction is unfinished.
Logical transaction is rejected; funds sent back.
Drive-by funding: someone saw the ephemeral address and sent a (maybe intentionally identifiable) amount of funds to it.

Each state needs to be handled differently; some states have multiple possible resolutions. None of this has been discussed or planned yet, and I don't think we have time to do so for 1.0.

I completely agree that an ideal UX would distinguish those (except possibly that the refund and drive-by cases might be indistinguishable).
Where I disagree is that I don't think any of them need to result in any possibility of funds loss. Just allocate the ephemeral address in a way that guarantees that we scan it.

Where I disagree is that I don't think any of them need to result in any possibility of funds loss.

I agree, and am not arguing for this!

Just allocate the ephemeral address in a way that guarantees that we scan it.

I agree with "Just allocate the ephemeral address in a way that guarantees that we can scan it" - this is sufficient to guarantee no loss of funds. I disagree with actually scanning it in 1.0, because per my comment above that would require UX work to explain to the user what is going on in all of the different states that scan might occur from, that we do not have time for.
Only one of the four states I outlined is transient (it decays to the second state after about an hour), but three of the states are permanent, distinguishable, and "automatically re-shield" is not necessarily the correct resolution to all three.
For 1.0, informing the user and telling them to contact us means that a) we don't make permanent changes to their wallet history that might be undesirable, and b) means that we learn about these edge cases if they occur, which helps us to craft better UX here.


Add credentials config that supports custom keys

Closes #606 #607
Code Changes

[x] Create new cli options for okta, aws. CLI commands now support credentials as options, fides config reference, and environment variables.
[x] Add helper functions to share code that validates and fetches credentials from options vs config. Not sure if click has some better ways to deal with options like this. see cli.utils.py
[x] Implement our own merging of environment variables that are prefixed with FIDESCTL__CREDENTIALS__. Does not use pydantic features because it doesn't work well with custom fields.  see credentials_settings.py
[x] Created models DatabaseCredentials/OktaCredentials/AwsCredentials which are only used for validation when configs are retrieved. see credentials_settings.py
[x] Update all functions in okta/aws code paths to allow passing in credentials directly from cli.

Steps to Confirm

[x] Manually tested [scan/generate] x [db/aws/okta] with options, environment variables, and config values
[x] Added cli integration tests which test the usage of new options and usage of credentials from config
[x] Added test cases for merging of environment variable config
[x] Added test cases for handling new options and having multiple ways to pass credentials now

Pre-Merge Checklist

[x] All CI Pipelines Succeeded
Documentation Updated:

[x] documentation complete, or draft/outline provided (tag docs-team to complete/review on this branch)
[x] documentation issue created (tag docs-team to complete issue separately)

[x] Issue Requirements are Met
[x] Relevant Follow-Up Issues Created
[x] Update CHANGELOG.md

Description Of Changes
First attempt was a little too complicated and got some help from @ThomasLaPiana to brainstorm a new solution. Will be closing #673 in favor of this PR. This PR should be simpler but it still took alot of effort to flesh out.
With this PR we now support sourcing credentials from command options or fides config reference for db/aws/okta commands.
Sample Usage/Output
Success with connection string option
Database generate using connection string
fidesctl generate dataset db dataset.yml --connection-string "redshift+psycopg2://fidesctl:$REDSHIFT_FIDESCTL_PASSWORD@redshift-cluster-1.cohs2e5eq2e4.us-east-1.redshift.amazonaws.com:5439/fidesctl_test"

Generated dataset manifest written to dataset.yml

Database generate using credentials-id option from env
export FIDESCTL__CREDENTIALS__MY_REDSHIFT__CONNECTION_STRING="redshift+psycopg2://fidesctl:$REDSHIFT_FIDESCTL_PASSWORD@redshift-cluster-1.cohs2e5eq2e4.us-east-1.redshift.amazonaws.com:5439/fidesctl_test"
fidesctl generate dataset db dataset.yml --credentials-id "my_redshift"

Generated dataset manifest written to dataset.yml

Database generate using credentials-id option from config
~inside fides.toml~
[credentials]
redshift_toml_1 = {connection_string="redshift+psycopg2://fidesctl:.wm!pV2kMc4ViTYE@redshift-cluster-1.cohs2e5eq2e4.us-east-1.redshift.amazonaws.com:5439/fidesctl_test"}

fidesctl generate dataset db dataset.yml --credentials-id "redshift_toml_1"

Generated dataset manifest written to dataset.yml

AWS generate using new secret_access_key/access_key_id options
fidesctl generate system aws manifest.yml --access_key_id $AWS_ACCESS_KEY_ID --secret_access_key $AWS_SECRET_ACCESS_KEY --region $AWS_DEFAULT_REGION```

Generated system manifest written to manifest.yml

AWS generate using credentials-id option from env
export FIDESCTL__CREDENTIALS__AWS_1__AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY_ID
export FIDESCTL__CREDENTIALS__AWS_1__AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEY
export FIDESCTL__CREDENTIALS__AWS_1__REGION_NAME="us-east-1"
fidesctl generate system aws manifest.yml --credentials-id aws_1

Generated system manifest written to manifest.yml

AWS generate using credentials-id option from config
~inside fides.toml~
[credentials]
aws_1 = {aws_access_key_id="redacted", aws_secret_access_key="redacted", region_name="us-east-1"}

fidesctl generate system aws manifest.yml --credentials-id aws_1

Generated system manifest written to manifest.yml

@earmenda this looks really good to me! Focused some testing around defining a credential ID for aws and populating it partially with env vars and partially in fidsctl.toml and triggered some helpful Client error messages from boto 👍🏽
I do think it might be worthwhile to have a follow on issue (or add in here) to provide either further information around how to use env vars with credential IDs in configuration.md or a user guide for some of it to help users avoid any potential pitfalls

@SteveDMurphy  ahh I  hadn't thought of making changes to configuration.md. It would be worthwhile to add a section in there regarding credentials section. Will add.
Thanks for the review :)

@conceptualshark @ThomasLaPiana @SteveDMurphy  Thanks for all the help in this PR. Just posted latest changes, this should be ready to be merged


Example fails with HTTP status code 400 (Bad Request)

Running the example throws the following exception:
org.springframework.web.client.HttpClientErrorException: 400 Bad Request
The reason is that Facebook's Graph API requires an access token for these requests.
Suggestion: Use (e.g.) Github's API by changing the requests generated from

http://graph.facebook.com/PivotalSoftware
http://graph.facebook.com/CloudFoundry
http://graph.facebook.com/SpringFramework

to

https://api.github.com/users/PivotalSoftware
https://api.github.com/users/CloudFoundry
https://api.github.com/users/Spring-Projects

Happy to provide a pull request if desired.

PR sounds great. Thanks!


[NEW]: Indie Developer selling commissioned pixel art as NFTs without permission from original artists

Date
2021-03-09
Description
Jason Rohrer, developer of indie game "The Castle Doctrine" announced his plan to auction 155 of the digital paintings that he commissioned for the game as NFTs on the OpenSea platform, without ever requesting permission from the original artists or informing them of his plan at all. In an email, Rohrer told Kotaku that he did not ask permission to sell people’s works as NFTs “mostly because having email conversations with 50+ people would exceed my bandwidth as a solo creator.”
Links
https://kotaku.com/game-artists-not-happy-that-developer-is-selling-their-1846465316
https://twitter.com/jasonrohrer/status/1369331652787400704
http://thecastledoctrine.net/crypto.php
Agreement

[X] I irrevocably agree to release my contribution under the CC-BY 3.0 License

Attribution name
No response
Personal link
No response

Thank you! https://web3isgoinggreat.com?id=2021-03-09-2


Document config changes introduced with Analytics Engine

Please also see this draft I made, I don't know if it's up-to-date anymore. Please validate, but maybe you can use some parts of it:
name: myConfigWithJanusgraphAndSpark
analyticsEngine:
  # enabled configures whether an analytics engine should be used.
  # Setting this to true leads to the options "useAnalyticsEngine" and
  # "forceRecalculate" to become available in the GraphQL GetMeta->Kind->Class
  # and Aggregate->Kind->Class.
  #
  # Important: If enabled is set to true, you must also configure an
  # analytics engine in your database connector. If the chosen connector
  # does not support an external analytics engine, enabled must be set
  # to false.
  enabled: true

  # defaultUseAnalyticsEngine configures what the "useAnalyticsEngine"
  # in the GraphQL API will default to when not set.
  defaultUseAnalyticsEngine: true
database:
  name: janusgraph
  database_config:
    url: http://janus:8182
    analyticsEngine:
      # enabled configures whether an analytics engine for this particular connector
      # is used.
      # Important: If enabled is set to true, the overall configuration property
      # analyticsEngine.enabled must also be set to true.
      enabled: true

      # type configures which analytics engine is used with this connector
      # currently the janusgraph connector supports only 'spark-analytics-api'
      type: spark-analytics-api

      # url to discover the specific analytics backend
      url: http://analytics-api:8080

Check, thanks @etiennedi. I'll pick this up asap.


The  CLI --password  argument is not recognized when trying to create an app.

The
az ad app create --display-name "appName" --password "password" --available-to-other-tenants
Response messaging
unrecognized arguments: -password "password" --available-to-other-tenants

Hi @Gariantroll,
Can you confirm if you are using az cli 2.36.0?

Hi @Gariantroll ,
nvrmind above answer.
Provision and publish a bot documentation got updated last week.
For az cli 2.39.0, the password parameter is no longer needed, Reference here.
Example:
az ad app create --display-name "<name>" --sign-in-audience "AzureADandPersonalMicrosoftAccount"
Let me know if you have any questions. Thanks.

Thanks,
I ended up using composer bot to make the bot and made the credentials and resources.
Thank you,
Alexis

From: Ram Fattah @.>
Sent: Monday, August 22, 2022 12:55 AM
To: MicrosoftDocs/bot-docs @.>
Cc: Gariantroll @.>; Mention @.>
Subject: Re: [MicrosoftDocs/bot-docs] The CLI --password argument is not recognized when trying to create an app. (Issue #2218)
Hi @Gariantrollhttps://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FGariantroll&data=05|01||2d8a443a18ff49eacb7a08da83fa8dca|84df9e7fe9f640afb435aaaaaaaaaaaa|1|0|637967409386775622|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000|||&sdata=VLB7zax1S2wppA%2B%2Fqnz8Tx61aujbIdPlFNazDjGEIwg%3D&reserved=0 ,
nvrmind above answer.
Provision and publish a bothttps://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fazure%2Fbot-service%2Fprovision-and-publish-a-bot%3Fview%3Dazure-bot-service-4.0%26tabs%3Duserassigned%252Ccsharp&data=05|01||2d8a443a18ff49eacb7a08da83fa8dca|84df9e7fe9f640afb435aaaaaaaaaaaa|1|0|637967409386775622|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000|||&sdata=zSV%2BajMkZt5BPWOJiPHVHMArVZJhfbwbcC6zdMZ5kSE%3D&reserved=0 documentation got updated last week.
For az cli 2.39.0https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FAzure%2Fazure-cli%2Freleases%2Ftag%2Fazure-cli-2.39.0&data=05|01||2d8a443a18ff49eacb7a08da83fa8dca|84df9e7fe9f640afb435aaaaaaaaaaaa|1|0|637967409386775622|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000|||&sdata=bGsItZm%2BiA2cgSxqBeiWM2B5tgSbT3%2Fs5wpBWQxkoVM%3D&reserved=0, the password parameter is no longer needed, Reference herehttps://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fcli%2Fazure%2Fad%2Fapp%3Fview%3Dazure-cli-latest%23az-ad-app-create%3A~%3Atext%3DUpdate%2520an%2520application.-%2Caz%2520ad%2520app%2520create%2Caz%2520ad%2520app%2520create%2520--display-name%2520mytestapp%2C-Create%2520an%2520application&data=05|01||2d8a443a18ff49eacb7a08da83fa8dca|84df9e7fe9f640afb435aaaaaaaaaaaa|1|0|637967409386775622|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000|||&sdata=fKXQ%2Feh%2FLdx3mpvBJ9bO1J33lfizF5xySVX8VNodkRM%3D&reserved=0.
Example:
az ad app create --display-name "" --sign-in-audience "AzureADandPersonalMicrosoftAccount"
Let me know if you have any questions. Thanks.
—
Reply to this email directly, view it on GitHubhttps://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FMicrosoftDocs%2Fbot-docs%2Fissues%2F2218%23issuecomment-1221822492&data=05|01||2d8a443a18ff49eacb7a08da83fa8dca|84df9e7fe9f640afb435aaaaaaaaaaaa|1|0|637967409386775622|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000|||&sdata=yBm%2FMoavmj2hveHYFQSTSVz8hcjYHz%2BQ5oXNgLaBXek%3D&reserved=0, or unsubscribehttps://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FABBF7DWSBKR2UIPZ6D767PLV2MB4RANCNFSM534X3ITA&data=05|01||2d8a443a18ff49eacb7a08da83fa8dca|84df9e7fe9f640afb435aaaaaaaaaaaa|1|0|637967409386775622|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000|||&sdata=s0jawnhcvpUCSklpzMVPxl6R8JndtXMpVepk0DqjK4U%3D&reserved=0.
You are receiving this because you were mentioned.Message ID: @.***>

Awesome!
Closing this issue for now. Please feel free to re-open if needed. Thanks.


"Călățele", "Călata", "Dealu Negru", "Finciu", "Văleni" are listed in the drop-down menu

Description:
When user writes "Călățele", "Călata", "Dealu Negru", "Finciu" and "Văleni" the locations appear in the drop-down menu next to the comune and the county which is a part according to the law
Precondition:
The website is up an running.

Step 1
Write in the search bar "Călățele"
Expected results
The location appears in the drop-down menu as "Sat Călățele, CLUJ (Călățele)"

Step 2
Press "x" button.
Expected results
The location was deleted from the search bar.

Step 3
Write in the search bar "Călata"
Expected results
The location appears in the drop-down menu as "Sat Călata, CLUJ (Călățele)"

Step 4
Press "x" button.
Expected results
The location was deleted from the search bar.

Step 5
Write in the search bar "Dealu Negru"
Expected results
The location appears in the drop-down menu as "Sat Dealu Negru, CLUJ (Călățele)"

Step 6
Press "x" button.
Expected results
The location was deleted from the search bar.

Step 7
Write in the search bar "Finciu"
Expected results
The location appears in the drop-down menu as "Sat Finciu, CLUJ (Călățele)"

Step 8
Press "x" button.
Expected results
The location was deleted from the search bar.

Step 9
Write in the search bar "Văleni"
Expected results
The location appears in the drop-down menu as "Sat Văleni, CLUJ (Călățele)"


Rename ./go test server, move tests, improve ./go setup and ./go test

The structure and scope of the tests and their accompanying scripts are now easier to follow. scripts/test is more streamlined and robust. ./go lint and ./go test are out of the
critical section of ./go setup, and logging across both ./go setup and ./go test is much improved.

Coverage remained the same at 97.906% when pulling 5d07e36475008751bbaa9c82231df44de8c0a1c9 on move-tests into d88a1876097a1de3713fef0d30cabad7fc285d8d on master.


Variable bottom drag...

Often bottom drag co-efficients depend on seafloor roughness and ocean stratification (i.e. lee-wave models, non-propagating drag etc), and of course these things vary spatially (and stratification can evolve in time) (usually something like D = - U C_L  -  U | U | C_Q, where C_L \sim N h^2/L and C_Q \sim h / L, where h is a height scale, L is a lateral length scale, and N the Brunt-Vaisala freq)
I'm currently writing a little package var_bot_drag that will mom_u/v_botdrag_coeff and allow var_bot_dragu_linear, var_bot_dragu_quadratic, var_bot_dragv_linear, var_bot_dragv_quadratic to be specified by files the same size as the Depth field.  I'm posting this for any comments on this as a possible new official package if it all works out, or to hear if it has already been done, and save myself some effort.
u and v drags are specified separately to partially account for anisotropic drag co-efficients. This gets complicated and maybe we will want something more complicated eventually, but this seems easy enough for now. It is a bit wasteful to read in both for isotropic drags, but the fields are just 2-D.
Time dependence is a little more complicated, as the linear drag often scales with N.  As a follow -on, the drag could be updated based on N.
Thanks for any thoughts.

@jklymak with pkg/ctrl, there is an option to specify a 2-D map of some (linear) drag coefficient. If you just want to provide a 2-D
map for this coefficient, it could be made available even outside pkg/ctrl. But for a more realistic drag that depends on stratification,
I think a new (small) package make more sense.

I am mentioning (draft) PR #574 where I tried to implement a quadratic bottom drag according to the law of the wall. I guess there's a lot of overlap with your efforts. @jklymak maybe it would be good if you created a new (draft) PR on which we then work together?


Upgrade Eclipse Memory Analyzer to 1.6.0

Eclipse MAT 1.6.0 was released a few weeks ago. This PR upgrades the dowloaded version.
Note that the app name changed in the zip, also for some reason the downloaded file had the .zip&r=1 extension for this reason I moved the r query param before the file query param in the download URI.
Changes to a cask
Editing an existing cask

[X] Commit message includes cask’s name (and new version, if applicable).
[X] brew cask audit --download {{cask_file}} is error-free.
[X] brew cask style --fix {{cask_file}} left no offenses.

I don't know why the build fail for style issue I did the brew cask style --fix :
.../homebrew-cask ± ❯ git log -1
commit f017de614324766df199bfd1530476a8cbf41fd4
Author: Brice Dutheil<EMAIL_ADDRESS>Date:   Sat Jul 23 16:23:36 2016 +0200

    Upgrade Eclipse Memory Analyzer to 1.6.0
.../homebrew-cask ± ❯ brew cask style --fix memoryanalyzer

1 file inspected, no offenses detected

Thanks for the review !

This needs a rebase.

Done


Screensharing icon shown as enabled by default

Was exploring the UIKit and realized that the Screensharing icon is shown as enabled even when the screen sharing isn't enabled.

This is the intended behaviour - If you enable  screensharing the icon changes to a cross icon to allow stopping the screenshare. This behaviour can be customized with your own icons.

Yes, had observed the same later. Since the other icons follow a different pattern, this appeared like an issue.

This behaviour can be customized with your own icons.
That's good to know.

There seems to be something unusual about screen sharing though. It doesn't work. Code is similar to what Getting Started/Beginner's (name could be different) guide suggests. After a more detailed exploration will file an issue if required.
In case there is a known issue and this helps in debugging, here is the code of App.js
import React, {useState} from 'react';
import AgoraUIKit from 'agora-react-uikit';

const App = () => {
	const [videoCall, setVideoCall] = useState(true);
	//get user id from query params.
	const userId = new URLSearchParams(window.location.search).get('userId');

	let token;
	if (userId == 1) {
		token = '007eJxTYPC+JFJSHnVMVMLQ17FBPHO7rH7pkS2GHEHPWSfYL14lNV2BwcDI2NQ8ydDY0MzUwsTALNXCJNnY0NTENMk8JdE4zdDCzWNpckMgI0P1HVVGRgYIBPHZGUpSi0sy89IZGAB11111';
	} else {
		token = '007eJxTYNi04XrV0xyLhqM+E2QFTuoas9f7l+ulrlyzIs1IYsdex+UKDAZGxqbmSYbGhmamFiYGZqkWJsnGhqYmpknmKYnGaYYW/J5LkxsCGRmetiWzMDJAIIjPzlCSWlySmZfOwAAAIt1S3Tcc';
	}

	const rtcProps = {
		appId: '66957b131658406e84c31545b7daa234',
		channel: 'testing', // your agora channel
		token: token,
		enableScreensharing: true,
	};

	const callbacks = {
		EndCall: () => setVideoCall(false),
	};

	return videoCall ? (

			<AgoraUIKit rtcProps={rtcProps} callbacks={callbacks} />

	) : (

 setVideoCall(true)}>Start Call

	);
};

export default App;


CropBox gets resized if image aspectRatio is wider than canvas aspect ratio

I am using the following settings:
autoCropArea: 1,
modal: false,
guides: false,
highlight: false,
background: false,
movable: false,
resizable: false,
rotatable: false,
aspectRatio: 3,
minCropBoxWidth: 1200,
minCropBoxHeight: 400,
minContainerWidth: 1200,
minContainerHeight: 400,
minCanvasWidth: 1200,
minCanvasHeight: 400
When loading an image, the image is zoomed in/out to fill the full width of the container.
If the aspect ration of the image is wider than the AR of the canvas, the image doesn't completely fill the canvas. If strict mode is active, this behaviour will resize the cropBox.
If the image AR is wider than the canvas AR, the image should be zoomed to 100% of the container HEIGHT instead of WIDTH to fill the container, so that the canvas and cropbox can stay at full size
I couldn't completely figure out where this behaviour is defined... probably somewhere in renderCanvas and/or limitCanvas
I tried to work around this issue by manually zooming in inside the built callback and then setting the cropBox to full size, but this doesn't work as intendes (as described in #293)

Thanks! There is an error when computes the canvas.minWidth/Height sizes.

Thanks, looks good now


Use a magenta hexagon instead of 🔹 in the PS1

... because of its likeness to the Toolbox logo. Note that the magenta
foreground colour is requested through a terminal escape sequence with
SGR parameters [1]. The specific colour code for magenta is 35.
[1] https://en.wikipedia.org/wiki/ANSI_escape_code

Committed as de67ff4bcc09a4d667943bbae1a98d34d3de9f1d
I don't know why it's showing as closed.


⚠ Bofrost.lu has degraded performance

In 1bb09df, Bofrost.lu (https://www.bofrost.lu) experienced degraded performance:

HTTP code: 200
Response time: 1037 ms

Resolved: Bofrost.lu performance has improved in ba9084f after 6 minutes.


Vulkan: "The vertex format used to create the pipeline does not match the vertex format bound." error appears randomly at model load

Godot version
4.0dev (1f69582)
System information
Windows 10, Vulkan, RTX 2080
Issue description
Sometimes I load a 3D model and I see this repeated in the console:
drivers/vulkan/rendering_device_vulkan.cpp:7201 - Condition "!vertex_array" is true.
Vertex amount (22901) must be a multiple of the amount of vertices required by the render primitive (3).
drivers/vulkan/rendering_device_vulkan.cpp:7201 - Condition "!vertex_array" is true.
The vertex format used to create the pipeline does not match the vertex format bound.

This is usually accompanied by one of the materials in the scene not loading.
Steps to reproduce
Unfortunately this does not happen consistently. If I load models in the background using ResourceLoader.load_threaded_request and add them to the scene one by one, it seems to be random as to which model this error occurs on. But it always starts showing the moment one of the models load.
I'm not sure where to start in terms of debugging this.
Minimal reproduction project
No response

@everythingandnothingdev Please upload a minimal reproduction project to make this easier to troubleshoot.

It's not minimal but I can reproduce the errors with this demo:
https://github.com/everythingandnothingdev/godot-blender-2.81-splash

That could be related but in that case those errors show consistently when the VoxelGI node is enabled, not related to model loading. Worth looking into either way.

I get similar errors as @akien-mga when VoxelGI is used (f3ab09413), the console is flooded, and CPU time in the editor nosedives.

Weirdly, neither the errors nor perf hit occur when previewing a camera.

When opening the project in https://github.com/godotengine/godot/commit/c80540f9a20b45d107b9bed1e749571a83492337, I get the following backtrace (even after removing the .godot/ folder that was committed to version control):
#0  StringName::StringName (this=0x7fffa0021000) at ./core/string/string_name.h:72
#1  0x0000000004d3625b in CowData<StringName>::resize<false> (this=0x7fffa6ffb630, p_size=1668505455) at ./core/templates/cowdata.h:310
#2  0x0000000004d35d3f in Vector<StringName>::resize (this=0x7fffa6ffb628, p_size=1668505455) at ./core/templates/vector.h:91
#3  0x00000000096dd9da in ResourceLoaderBinary::open (this=0x7fffa6ffb5c8, p_f=..., p_no_resources=false, p_keep_uuid_paths=true) at core/io/resource_format_binary.cpp:1025
#4  0x00000000096de5c3 in ResourceLoaderBinary::get_dependencies (this=0x7fffa6ffb5c8, p_f=..., p_dependencies=0x7fffa6ffb850, p_add_types=false) at core/io/resource_format_binary.cpp:924
#5  0x00000000096df4ef in ResourceFormatLoaderBinary::get_dependencies (this=0xa921030, p_path=..., p_dependencies=0x7fffa6ffb850, p_add_types=false)
    at core/io/resource_format_binary.cpp:1185
#6  0x00000000096fc55a in ResourceLoader::get_dependencies (p_path=..., p_dependencies=0x7fffa6ffb850, p_add_types=false) at core/io/resource_loader.cpp:731
#7  0x000000000654d6fe in EditorFileSystem::_get_dependencies (this=0xf1c9260, p_path=...) at editor/editor_file_system.cpp:1470
#8  0x0000000006548b85 in EditorFileSystem::_scan_new_dir (this=0xf1c9260, p_dir=0x7fffa0002ee0, da=..., p_progress=...) at editor/editor_file_system.cpp:905
#9  0x0000000006547927 in EditorFileSystem::_scan_new_dir (this=0xf1c9260, p_dir=0x7fffa0001360, da=..., p_progress=...) at editor/editor_file_system.cpp:792
#10 0x0000000006547927 in EditorFileSystem::_scan_new_dir (this=0xf1c9260, p_dir=0x7fffa0000f40, da=..., p_progress=...) at editor/editor_file_system.cpp:792
#11 0x0000000006547109 in EditorFileSystem::_scan_filesystem (this=0xf1c9260) at editor/editor_file_system.cpp:320
#12 0x000000000654a34d in EditorFileSystem::_thread_func (_userdata=0xf1c9260) at editor/editor_file_system.cpp:346
#13 0x000000000955653f in Thread::callback (p_self=0xf1c9668, p_settings=..., p_callback=0x654a330 <EditorFileSystem::_thread_func(void*)>, p_userdata=0xf1c9260) at core/os/thread.cpp:75
#14 0x00000000095573ea in std::__invoke_impl<void, void (*)(Thread*, Thread::Settings const&, void (*)(void*), void*), Thread*, Thread::Settings, void (*)(void*), void*> (
    __f=@0x1f75ce68: 0x95564c0 <Thread::callback(Thread*, Thread::Settings const&, void (*)(void*), void*)>, __args=@0x1f75ce48: 0xf1c9260, __args=@0x1f75ce48: 0xf1c9260,
    __args=@0x1f75ce48: 0xf1c9260, __args=@0x1f75ce48: 0xf1c9260) at /usr/bin/../lib/gcc/x86_64-redhat-linux/12/../../../../include/c++/12/bits/invoke.h:61
#15 0x0000000009557261 in std::__invoke<void (*)(Thread*, Thread::Settings const&, void (*)(void*), void*), Thread*, Thread::Settings, void (*)(void*), void*> (
    __fn=@0x1f75ce68: 0x95564c0 <Thread::callback(Thread*, Thread::Settings const&, void (*)(void*), void*)>, __args=@0x1f75ce48: 0xf1c9260, __args=@0x1f75ce48: 0xf1c9260,
    __args=@0x1f75ce48: 0xf1c9260, __args=@0x1f75ce48: 0xf1c9260) at /usr/bin/../lib/gcc/x86_64-redhat-linux/12/../../../../include/c++/12/bits/invoke.h:96
#16 0x00000000095571dd in std::thread::_Invoker<std::tuple<void (*)(Thread*, Thread::Settings const&, void (*)(void*), void*), Thread*, Thread::Settings, void (*)(void*), void*> >::_M_invoke<0ul, 1ul, 2ul, 3ul, 4ul> (this=0x1f75ce48) at /usr/bin/../lib/gcc/x86_64-redhat-linux/12/../../../../include/c++/12/bits/std_thread.h:252
#17 0x0000000009557145 in std::thread::_Invoker<std::tuple<void (*)(Thread*, Thread::Settings const&, void (*)(void*), void*), Thread*, Thread::Settings, void (*)(void*), void*> >::operator() (this=0x1f75ce48) at /usr/bin/../lib/gcc/x86_64-redhat-linux/12/../../../../include/c++/12/bits/std_thread.h:259
#18 0x0000000009556db9 in std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (*)(Thread*, Thread::Settings const&, void (*)(void*), void*), Thread*, Thread::Settings, void (*)(void*), void*> > >::_M_run (this=0x1f75ce40) at /usr/bin/../lib/gcc/x86_64-redhat-linux/12/../../../../include/c++/12/bits/std_thread.h:210
#19 0x000000000a04dea3 in execute_native_thread_routine ()
#20 0x00007ffff7a8ce2d in start_thread () from /lib64/libc.so.6
#21 0x00007ffff7b121b0 in clone3 () from /lib64/libc.so.6

Built-in crash handler fails to kick in, so I used gdb here.


Blocking motorways fails

When using a rather small block_area this blocks only one of the directions as the ID search stops when going outside of the boundary and just adds one edgeId. We could either make the current 1km² setting changable or properly find all edgeIds in this bounding box.
The shape method has additionally the disadvantage that if the junction nodes are outside of the bounding box (but the edge is still crossing the bbox that happens frequently with motorways or tunnels) the edge is not skipped. But fetching the edge geometry could be too expensive.

Could you add an example?
Another issue for motorways could be that we use BFS to find "all" edges in a shape. If there is no connection from one edge to another in the shape, our current approach might fail.
At the time of creating the block feature, we haven't had the findNClosest method in the LocationIndexTree. Maybe we could use this method in combination with an EdgeFilter that filters Edges that lie outside of the shape?

Yeah, an example is important :)
This should block via edge ids and this route via shape but both do not work at the moment.
This route is properly blocked where the edge id search is or this route that includes the tower node in the block_area.
I'm unsure why this route fails even for the opposite direction as the edge id search should find at least one of the two edges.

If there is no connection from one edge to another in the shape, our current approach might fail.

Yes, that was what I meant :)

Maybe we could use this method in combination with an EdgeFilter that filters Edges that lie outside of the shape?

How do we detect if an edge is outside of a shape? If one or two tower nodes are outside of the shape the edge could be still inside and if we fetch the whole geometry we have to take care of the performance. Maybe a good compromise is: if the shape is small call fetchWayGeometry and if the shape is big assume there will be at least one tower node of every edge inside the shape. And we do a performance comparison in a city (as it is an edge-dense area) which area size should be used as new default.

Maybe a good compromise is: if the shape is small then call fetchWayGeometry and if the shape is big then assume there will be at least one tower node of every edge inside the shape and use the shape contains method. And we do a performance comparison in a city (as it is an edge-dense area) which area size should be used as new default.

How about, if the road is longer than half the shape length, then we fetch the geometry (or something like this)?

We have to see if distance calculation is really faster then fetching the full way geometry ;)

I think there are currently 3 very similar things in GH but handled completely different with their own limitations and especially the block_area has this bug and #1127 so it might be interesting to think about this again:

spatial rules creates an index before import to handle country specific speeds and access properties (also polygons are supported but resolution is per default not too fine)
the ChangeGraphHelper uses a JSON from a request to change speeds and access properties (bbox, line, multiple points)
this block_area parameter changes access properties (currently only bbox, point and circle)

Closing in favor of #1324


monadic Parsed

It would be very useful to have some monadic functions to run on parsed result (in a similar way like we work with Try-s), things: like map, recover, orElse, getOrElse, etc

@antonkulaga great idea :) Maybe I will implement it in a month or so.


Agree on peer review table layout

Example/Template (note that you may adjust the list of items below depending on the nature of your project. However, please use an agreed list for the whole team)

An agreed upon format will be used in the final peer and self review docs.


Document the CLI and its options

I.e., -p and all that. The old docs likely have bits to port over.

See also discussion in #820.

I would love to have cli alternative to configs (where applicable) in the docs too. I don't tend to write config file for quick POC or just for hacking. +1


Attach multiple interfaces for same network

I have a fairly simple use case: how to create secondary interface in the POD and attach it to the same network as the default interface?
My deployment is EKS. I'm using Calico. AWS CNI completely removed
Relevant multus config:
kind: ConfigMap
apiVersion: v1
metadata:
  name: multus-cni-config
  namespace: kube-system
  labels:
    tier: node
    app: multus
data:
  # NOTE: If you'd prefer to manually apply a configuration file, you may create one here.
  # In the case you'd like to customize the Multus installation, you should change the arguments to the Multus pod
  # change the "args" line below from
  # - "--multus-conf-file=auto"
  # to:
  # "--multus-conf-file=/tmp/multus-conf/70-multus.conf"
  # Additionally -- you should ensure that the name "70-multus.conf" is the alphabetically first name in the
  # /etc/cni/net.d/ directory on each node, otherwise, it will not be used by the Kubelet.
  cni-conf.json: |
    {
      "name": "multus-cni-network",
      "type": "multus",
      "cniVersion": "0.3.1",
      "LogLevel": "debug",
      "LogFile": "/var/log/multus.log",
      "confDir": "/etc/cni/net.d/",
      "readinessindicatorfile": "/etc/cni/net.d/10-calico.conflist",
      "capabilities": {
        "portMappings": true
      },
      "delegates": [
        {
          "cniVersion": "0.3.1",
          "name": "default-cni-network",
          "plugins": [
              {
                           "name": "k8s-pod-network",
                           "cniVersion": "0.3.1",
                           "type": "calico",
                           "log_level": "info",
                           "datastore_type": "kubernetes",
                           "nodename": "__KUBERNETES_NODE_NAME__",
                           "mtu": 1440,
                           "ipam": {
                             "type": "calico-ipam"
                           },
                           "policy": {
                             "type": "k8s"
                           },
                           "kubernetes": {
                             "kubeconfig": "/etc/cni/net.d/calico-kubeconfig"
                           }
              },
              {
                "type": "portmap",
                "snat": true,
                "capabilities": {
                  "portMappings": true
                }
              }
          ]
        }
      ],
      "kubeconfig": "/etc/cni/net.d/multus.d/multus.kubeconfig"
    }

DaemonSet relevant part:
      containers:
      - name: kube-multus
        image: docker.io/nfvpe/multus:stable
        env:
        - name: KUBERNETES_NODE_NAME
          valueFrom:
            fieldRef:
              fieldPath: spec.nodeName
        command:
          - /bin/bash
          - -cex
          - |
            #!/bin/bash
            sed "s|__KUBERNETES_NODE_NAME__|${KUBERNETES_NODE_NAME}|g" /tmp/multus-conf/00-multus.conf.template > /tmp/00-multus.conf
            /entrypoint.sh \
              --multus-conf-file=/tmp/00-multus.conf

I've created testing purpose:
apiVersion: "k8s.cni.cncf.io/v1"
kind: NetworkAttachmentDefinition
metadata:
  name: calico-conf
  namespace: kube-system
spec:
  config: '{
    "cniVersion": "0.3.1",
    "type": "calico",
    "log_level": "info",
    "datastore_type": "kubernetes",
    "nodename": "ip-10-11-2-68.eu-west-1.compute.internal",
    "mtu": 1440,
    "ipam": {
      "type": "calico-ipam"
    },
    "policy": {
      "type": "k8s"
    },
    "kubernetes": {
      "kubeconfig": "/etc/cni/net.d/calico-kubeconfig"
    }
  }'

First question: I understand that no matter the fact that the default network calico-config is present somewhere in kube-system ConfigMap I still have to create NetworkAttachmentDefinition?
And the POD, relevant fragment
apiVersion: v1
kind: Pod
metadata:
  annotations:
    k8s.v1.cni.cncf.io/networks: kube-system/calico-conf
  labels:
    role: debug
    app: application
  name: app

The POD actually starts (logs show that actually two IPs were assigned) and immediately completes (!)
Digging in logs showed something about eth0 going down (during second IP request)
Second question what should I change in order to start my pod with two IPs (routable across the cluster!)?
Third question does anyone has working example of multi interface POD setup? Not host-local
Fourth question Why all examples use host-local?! How to make it route across the cluster (across k8s nodes)? Without routing this is completely useless feature

During interface add into the pod, multus just invokes CNI plugin and multus does NOT touch network interface.

Digging in logs showed something about eth0 going down (during second IP request)

So I guess that second calico CNI invocation may cause eth0 down. First, I guess calico just assume only one interface in the pod and calico does NOT assume two or more calico interface in pod with multus.
I am not good at calico so could you please ask someone else about it?
For now, as above reason, I think it is not multus issue, so close it.


Failure to start SQL trigger on .NET 7

Due to project dependencies, I'm working with a function app that targets .NET 7 on an isolated process on Azure functions v4.
On Friday, I started using this package as I wanted to make use of the SQL Triggers functionality. I had it working with a table for an initial test but on Monday I made some changes to use a new table. Since that, I have been dealing with an issue that I can't seem to figure out. So far the only fix I've found is switching to target .NET 6 (which requires commenting out a bunch of existing code in the project due to project dependencies). Unless I'm missing something I thought that .NET 7 is supported with this package, even though it targets .NET 6.
Here is the exception output:
Failed to start SQL trigger listener for table: '', function ID: ''. Exception: System.TypeInitializationException: The type initializer for 'Microsoft.Data.SqlClient.SqlAuthenticationProviderManager' threw an exception.
System.IO.FileNotFoundException: Could not load file or assembly 'System.Threading, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'. The system cannot find the file specified.
File name: 'System.Threading, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'For detailed output, run func with --verbose flag.
at System.Configuration.ConfigurationManager.EnsureConfigurationSystem()
at System.Configuration.ConfigurationManager.PrepareConfigSystem()
at System.Configuration.ConfigurationManager.GetSection(String sectionName)
at Microsoft.Data.SqlClient.SqlAuthenticationProviderManager.FetchConfigurationSection[T](String name)
at Microsoft.Data.SqlClient.SqlAuthenticationProviderManager..cctor()
--- End of inner exception stack trace ---
at Microsoft.Data.SqlClient.SqlInternalConnectionTds..ctor(DbConnectionPoolIdentity identity, SqlConnectionString connectionOptions, SqlCredential credential, Object providerInfo, String newPassword, SecureString newSecurePassword, Boolean redirectedUserInstance, SqlConnectionString userConnectionOptions, SessionData reconnectSessionData, Boolean applyTransientFaultHandling, String accessToken, DbConnectionPool pool)
at Microsoft.Data.SqlClient.SqlConnectionFactory.CreateConnection(DbConnectionOptions options, DbConnectionPoolKey poolKey, Object poolGroupProviderInfo, DbConnectionPool pool, DbConnection owningConnection, DbConnectionOptions userOptions)
at Microsoft.Data.ProviderBase.DbConnectionFactory.CreatePooledConnection(DbConnectionPool pool, DbConnection owningObject, DbConnectionOptions options, DbConnectionPoolKey poolKey, DbConnectionOptions userOptions)
at Microsoft.Data.ProviderBase.DbConnectionPool.CreateObject(DbConnection owningObject, DbConnectionOptions userOptions, DbConnectionInternal oldConnection)
at Microsoft.Data.ProviderBase.DbConnectionPool.UserCreateRequest(DbConnection owningObject, DbConnectionOptions userOptions, DbConnectionInternal oldConnection)
at Microsoft.Data.ProviderBase.DbConnectionPool.TryGetConnection(DbConnection owningObject, UInt32 waitForMultipleObjectsTimeout, Boolean allowCreate, Boolean onlyOneCheckConnection, DbConnectionOptions userOptions, DbConnectionInternal& connection)
at Microsoft.Data.ProviderBase.DbConnectionPool.WaitForPendingOpen()
--- End of stack trace from previous location ---
at Microsoft.Azure.WebJobs.Extensions.Sql.SqlBindingUtilities.OpenAsyncWithSqlErrorHandling(SqlConnection connection, CancellationToken cancellationToken)
at Microsoft.Azure.WebJobs.Extensions.Sql.SqlTriggerListener`1.StartAsync(CancellationToken cancellationToken)
Apologies if the issue is something trivial on my end or not related to this repo.
If so, any suggestions would be greatly appreciated.

Good day!  I have experienced a similar issue recently.  I get the same error even on a SqlInput trigger that was working.  I have a repo that demonstrates.
The key difference is the version of the Extensions.Sql chosen.  If I use the 2.x version no problem.  But as soon as I execute with a 3.X I get the error:
[2023-09-13T02:35:43.326Z] Executed 'Functions.GetToDos' (Failed, Id=d156bf2f-b79f-4e5a-9431-96ac4e1ce950, Duration=247ms) [2023-09-13T02:35:43.330Z] System.Private.CoreLib: Exception while executing function: Functions.GetToDos. Microsoft.Data.SqlClient: The type initializer for 'Microsoft.Data.SqlClient.SqlAuthenticationProviderManager' threw an exception. System.Configuration.ConfigurationManager: Could not load file or assembly 'System.Threading, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'. The system cannot find the file specified.
What is strange is another project that I had working about 10 days ago with all three SQL items, input, output, and bindings was working.  Then suddenly this weekend it stopped.  I have tried recreating a number of times and they still fail.
What has happened?

Do I understand that GA will only be for .Net 6?
I posted here as well as in the link MaddyDev has above.

We are working on getting a version of the extension out that rolls back the change which broke this to immediately unblock people, and then as Maddy said we will be working to find a long term fix that won't break .NET 7 as well for GA.

@keithnicholson Were you able to successfully run a trigger with .NET 7 and 3.0.341-preview package?

@MaddyDev I haven't been able to run a trigger on .NET 7 with the 3.0.341-preview package since last week I believe if that's any help. (Same error as first post)

@MaddyDev  I don't recall if it was 3.0.341 or 3.0.343.  I would imagine it was 3.0.343 since I had not started working on this until late August.  Sadly, I had not been committing the work to a local repo to validate which one I had before I updated every dependency.
I have tried taking it back to both of these and earlier one, yet still no difference.

@MaddyDev I haven't been able to run a trigger on .NET 7 with the 3.0.341-preview package since last week I believe if that's any help. (Same error as first post)

@Stxtics Thank you, just double checking again: you were able to successfully run a SQL trigger binding function with .NET 7 before? Or is it an input/output binding?

@MaddyDev I haven't been able to run a trigger on .NET 7 with the 3.0.341-preview package since last week I believe if that's any help. (Same error as first post)

@Stxtics Thank you, just double checking again: you were able to successfully run a SQL trigger binding function with .NET 7 before? Or is it an input/output binding?

@MaddyDev Yes the SQL trigger functionality. I believe it worked on Friday for an initial test. then on Monday I couldn't get it to work.

@Stxtics, @keithnicholson We have released a new package addressing the issue that's affecting .NET 7. This should be resolved, can you confirm if you're able to run a SQL trigger successfully. Thanks.

@maddydev
I'll give it a try today.  Thanks.
Best regards,
Keith Nicholson

From: Maddy @.>
Sent: Tuesday, September 26, 2023 2:09:21 PM
To: Azure/azure-functions-sql-extension @.>
Cc: Keith Nicholson @.>; Mention @.>
Subject: Re: [Azure/azure-functions-sql-extension] Failure to start SQL trigger on .NET 7 (Issue #920)
@Stxticshttps://github.com/Stxtics, @keithnicholsonhttps://github.com/keithnicholson We have released a new package addressing the issue that's affecting .NET 7. This should be resolved, can you confirm if you're able to run a SQL trigger successfully. Thanks.
—
Reply to this email directly, view it on GitHubhttps://github.com/Azure/azure-functions-sql-extension/issues/920#issuecomment-1736135409, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AARACMBXNTHFEFHWCFY4WODX4MR6DANCNFSM6AAAAAA4UW4ALA.
You are receiving this because you were mentioned.Message ID: @.***>

@MaddyDev not sure that I'll be able to test this soon so I'm happy for the issue to be closed if confident enough it has been fixed.

Closing this as fixed by the new release 3.0.433-preview

I was able to do a sqlinput.  It worked.  Having an issue with the sqloutput.  I'll get back to you tomorrow.

All works great with the latest.
Thank you!


[MINOR][DOC] Documentation on JVM options for SBT

What changes were proposed in this pull request?
Documentation and .gitignore
How was this patch tested?
Manual test that SBT honors the settings in .jvmopts if present

Is there any reason not to just check in a .jvmopts file that captures these settings, and remove the corresponding entry from .gitignore? Seems like that would be better, since it saves contributors from a manual step. If appropriate, I can submit a PR for this.


Opinion - supertest vs pactum

Issue Name
Opinion on supertest vs pactum
Summary
I tried out pactum while migrating test cases from the old backend to the new backend. I think the library is very good and also a lot easier to debug than supertest. One example could be when I want to see how the request I send to the backend looks like with supertest I need to console log and got a very huge object from supertest with all the things that might not related to the information I am looking for. With pactum I got the specific information I want to know. Looking at pactum's weekly download it doesn't have millions of downloads weekly like supertest but I think maybe it just because everyone is used to supertest. I create this issue to ask for opinion. If it is ok that I continue using pactum instead of supertest. If you know that supertest is better then let my know why since I haven't used supertest much and maybe the things I found inconvenient with supertest might be because I dont know the correct way to use it. @henrikjohansson712, @stephan271, @nitrosx, @minottic, @dylanmcreynolds.

@nguyenlinhlinh unfortunately, I do not have much experience with testing libraries in javascript. I would rely on your and the community expertise and experience to guide the decision.
That said, I'm not oppose to change testing library if you think the gain is worth it.
On this matter, I found this post where different testing libraries are compared:
https://dev.to/asaianudeep/api-testing-tools-in-javascript-22d8

I say we try it

I have no experience with pactum , but I am happy to give it a try

Ok, Thank you. I will close this then :)


Logging handler causes breakage interacting with other packages

I'm using pyxform as part of an asynchronous task with Celery. Celery performs some deep magic on the standard Python logging system when forking, and chokes since 07301f95
Celery attempts to start it's workers a few times before failing
Traceback (most recent call last):
  File "/home/user/opt/virtualenvs/project/lib/python2.7/site-packages/billiard/process.py", line 315, in _bootstrap
    handler.createLock()
TypeError: unbound method createLock() must be called with NullHandler instance as first argument (got nothing instead)
Process ForkPoolWorker-3:
Traceback (most recent call last):
  File "/home/user/opt/virtualenvs/project/lib/python2.7/site-packages/billiard/process.py", line 315, in _bootstrap
    handler.createLock()
TypeError: unbound method createLock() must be called with NullHandler instance as first argument (got nothing instead)
Process ForkPoolWorker-4:
Traceback (most recent call last):
  File "/home/user/opt/virtualenvs/project/lib/python2.7/site-packages/billiard/process.py", line 315, in _bootstrap
    handler.createLock()
TypeError: unbound method createLock() must be called with NullHandler instance as first argument (got nothing instead)
Process ForkPoolWorker-5:
Traceback (most recent call last):
  File "/home/user/opt/virtualenvs/project/lib/python2.7/site-packages/billiard/process.py", line 315, in _bootstrap
    handler.createLock()
TypeError: unbound method createLock() must be called with NullHandler instance as first argument (got nothing instead)
Process Beat:
Traceback (most recent call last):
  File "/home/user/opt/virtualenvs/project/lib/python2.7/site-packages/billiard/process.py", line 315, in _bootstrap
    handler.createLock()
TypeError: unbound method createLock() must be called with NullHandler instance as first argument (got nothing instead)

Fixed by #402


Webpack 4 Support?

Do we know if this plugin has support for webpack v4? I have a relatively simple setup and am getting the following error:
TypeError: compilation.fileDependencies.indexOf is not a function
    at fileDependencies.forEach.file (/....../node_modules/nunjucks-webpack-plugin/src/NunjucksWebpackPlugin.js:107:42)

Any ideas?

@burt202 need PR for solve this problem, PR welcome :+1:

PR created at https://github.com/itgalaxy/nunjucks-webpack-plugin/pull/9


VM freezes when I try to run anything

I had this working earlier, but now it reliably freezes and I can't get anything to work.
Steps taken:
# novm-import-kernel
# novm create
# novm list
# novm run --id <output from novm list> echo hi

I then observe the CPU get pegged in novmm. stracing reveals it's stuck on the KVM_RUN ioctl. I sent sigquit to the binary consuming 100% CPU, here's the trace. Do we expect so many IoHandlers in particular?
https://gist.github.com/pwaller/78271959acac8d97adcc
excerpt:
SIGQUIT: quit
PC=0x7ff2bdaaa867
signal arrived during cgo execution

goroutine 54 [syscall]:
runtime.cgocall(0x4049a0, 0x7ff2bc0c4e50)
	/usr/lib/go/src/pkg/runtime/cgocall.c:143 +0xe5 fp=0x7ff2bc0c4e38 sp=0x7ff2bc0c4df0
novmm/platform._Cfunc_kvm_run(0xa00000004, 0xc208026a78, 0x0)
	novmm/platform/_obj/_cgo_defun.c:443 +0x31 fp=0x7ff2bc0c4e50 sp=0x7ff2bc0c4e38
novmm/platform.(*Vcpu).Run(0xc208026840, 0x0, 0x0)
	/home/pwaller/.local/src/github.com/google/novm/src/novmm/platform/kvm_run.go:107 +0x12c fp=0x7ff2bc0c4e98 sp=0x7ff2bc0c4e50
main.Loop(0xc2080281c0, 0xc208026840, 0xc20801ad70, 0xc2093f4980, 0x0, 0x0)
	/home/pwaller/.local/src/github.com/google/novm/src/novmm/loop.go:34 +0xd2 fp=0x7ff2bc0c4f40 sp=0x7ff2bc0c4e98
main.func·001(0xc208026840)
	/home/pwaller/.local/src/github.com/google/novm/src/novmm/main.go:286 +0x55 fp=0x7ff2bc0c4fa0 sp=0x7ff2bc0c4f40
runtime.goexit()
	/usr/lib/go/src/pkg/runtime/proc.c:1445 fp=0x7ff2bc0c4fa8 sp=0x7ff2bc0c4fa0
created by main.main
	/home/pwaller/.local/src/github.com/google/novm/src/novmm/main.go:288 +0xd69

I've tried a variety of different programs specified by /bin/ls or just ls. I'm sure I had them working at some point but now they reliably freeze. I also deleted my .novm directory and started again, but this failed too.

Aha! If I specify --init to novm create then run works. But I can't figure out how to run things inside novm in any other way so far..

Yes, the goroutines are expected.
I was hoping that 302881d0ff0b3e13385a96b564731c768208b73b would help with this issue but it doesn't seem to. I've seen the spinning before, and I'll invest some time to track it down when I can. (There's a lot to do before novm is reliable!)

@pwaller
hello
what‘s your full comand？

I believe this issue is related to #23. Please reopen if it persists.


error about simple_parsing

Hello, thank you for your teams work, it's really helpful.
However, when I run this command roslaunch elevation_mapping_cupy elevation_mapping_cupy.launch in the terminal, I got this error :
terminate called after throwing an instance of 'pybind11::error_already_set'
  what():  ModuleNotFoundError: No module named 'simple_parsing'

At:
  /home/yc/catkin_map/catkin_eth_cupy/src/elevation_mapping_cupy/elevation_mapping_cupy/script/elevation_mapping_cupy/parameter.py(8): <module>
  <frozen importlib._bootstrap>(219): _call_with_frames_removed
  <frozen importlib._bootstrap_external>(848): exec_module
  <frozen importlib._bootstrap>(686): _load_unlocked
  <frozen importlib._bootstrap>(975): _find_and_load_unlocked
  <frozen importlib._bootstrap>(991): _find_and_load
  /home/yc/catkin_map/catkin_eth_cupy/src/elevation_mapping_cupy/elevation_mapping_cupy/script/elevation_mapping_cupy/__init__.py(1): <module>
  <frozen importlib._bootstrap>(219): _call_with_frames_removed
  <frozen importlib._bootstrap_external>(848): exec_module
  <frozen importlib._bootstrap>(686): _load_unlocked
  <frozen importlib._bootstrap>(975): _find_and_load_unlocked
  <frozen importlib._bootstrap>(991): _find_and_load
  <frozen importlib._bootstrap>(219): _call_with_frames_removed
  <frozen importlib._bootstrap>(961): _find_and_load_unlocked
  <frozen importlib._bootstrap>(991): _find_and_load

I have install simple_parsing with pip, but the error still exists. I still don't know the reason.
So do you know how to fix it? Thanks you for your reply.

You just need to install the package with pip install simple-parsing and not with --user.
The package should be then here:
/usr/local/lib/python3.8/dist-packages/simple_parsing

Tkank you for your help. However, it still doesn't wrok.
When I input this command:
Requirement already satisfied: simple_parsing in /home/yc/.local/lib/python3.11/site-packages (0.1.5)
Requirement already satisfied: docstring-parser~=0.15 in /home/yc/.local/lib/python3.11/site-packages (from simple_parsing) (0.15)
Requirement already satisfied: typing-extensions>=4.5.0 in /home/yc/anaconda3/lib/python3.11/site-packages (from simple_parsing) (4.9.0)

and when I launch the file, the error shows here:
terminate called after throwing an instance of 'pybind11::error_already_set'
  what():  ModuleNotFoundError: No module named 'simple_parsing'

At:
  /home/yc/catkin_map/catkin_eth_cupy/src/elevation_mapping_cupy/elevation_mapping_cupy/script/elevation_mapping_cupy/parameter.py(8): <module>
  <frozen importlib._bootstrap>(219): _call_with_frames_removed
  <frozen importlib._bootstrap_external>(848): exec_module
  <frozen importlib._bootstrap>(686): _load_unlocked
  <frozen importlib._bootstrap>(975): _find_and_load_unlocked
  <frozen importlib._bootstrap>(991): _find_and_load
  /home/yc/catkin_map/catkin_eth_cupy/src/elevation_mapping_cupy/elevation_mapping_cupy/script/elevation_mapping_cupy/__init__.py(1): <module>
  <frozen importlib._bootstrap>(219): _call_with_frames_removed
  <frozen importlib._bootstrap_external>(848): exec_module
  <frozen importlib._bootstrap>(686): _load_unlocked
  <frozen importlib._bootstrap>(975): _find_and_load_unlocked
  <frozen importlib._bootstrap>(991): _find_and_load
  <frozen importlib._bootstrap>(219): _call_with_frames_removed
  <frozen importlib._bootstrap>(961): _find_and_load_unlocked
  <frozen importlib._bootstrap>(991): _find_and_load

By the way, I use the Ubuntu20.04, cuda 12.4, ros-notetic

seems like you installed it with --user, so you have to uninstall it first, and reinstall it without the --user flag.
pip uninstall simple-parsing typing-extensions docstring-parser
and then install it again with pip install simple-parsing
should fix the problem.

Yes, when I use conda with python3.8, I reinstall simple-parsing, and it works.
Thanks again for your quick reply here.

Sorry again, when I launch roslaunch elevation_mapping_cupy turtlesim_semantic_pointcloud_example.launch and ````roslaunch elevation_mapping_cupy turtlesim_semantic_image_example.launch```, there also shows the error:
Traceback (most recent call last):
  File "/home/yc/catkin_map/catkin_eth/devel/lib/semantic_sensor/image_node.py", line 15, in <module>
    exec(compile(fh.read(), python_script, 'exec'), context)
  File "/home/yc/catkin_map/catkin_eth/src/elevation_mapping_cupy/sensor_processing/semantic_sensor/script/semantic_sensor/image_node.py", line 21, in <module>
    from semantic_sensor.image_parameters import ImageParameter
  File "/home/yc/catkin_map/catkin_eth/src/elevation_mapping_cupy/sensor_processing/semantic_sensor/script/semantic_sensor/image_parameters.py", line 2, in <module>
    from simple_parsing.helpers import Serializable
ModuleNotFoundError: No module named 'simple_parsing'
[front_cam-5] process has died [pid 191764, exit code 1, cmd /home/yc/catkin_map/catkin_eth/devel/lib/semantic_sensor/image_node.py front_cam_image __name:=front_cam __log:=/home/yc/.ros/log/4915dd76-dfc3-11ee-876d-89b345875324/front_cam-5.log].
log file: /home/yc/.ros/log/4915dd76-dfc3-11ee-876d-89b345875324/front_cam-5*.log

and
Traceback (most recent call last):
  File "/home/yc/catkin_map/catkin_eth/devel/lib/semantic_sensor/pointcloud_node.py", line 15, in <module>
    exec(compile(fh.read(), python_script, 'exec'), context)
  File "/home/yc/catkin_map/catkin_eth/src/elevation_mapping_cupy/sensor_processing/semantic_sensor/script/semantic_sensor/pointcloud_node.py", line 17, in <module>
    from semantic_sensor.pointcloud_parameters import PointcloudParameter
  File "/home/yc/catkin_map/catkin_eth/src/elevation_mapping_cupy/sensor_processing/semantic_sensor/script/semantic_sensor/pointcloud_parameters.py", line 2, in <module>
    from simple_parsing.helpers import Serializable
ModuleNotFoundError: No module named 'simple_parsing'
[front_cam-5] process has died [pid 187311, exit code 1, cmd /home/yc/catkin_map/catkin_eth/devel/lib/semantic_sensor/pointcloud_node.py front_cam __name:=front_cam __log:=/home/yc/.ros/log/72941ca0-dfc1-11ee-876d-89b345875324/front_cam-5.log].
log file: /home/yc/.ros/log/72941ca0-dfc1-11ee-876d-89b345875324/front_cam-5*.log

I wonder why this issue can exist, and how can I fix it?
Thanks again here.

I've solve this issue,
just install the commands:
sudo pip3 install simple_parsing
sudo pip3 install numpy==1.21.1
sudo apt-get install ros-noetic-ros-nump

I can launch both command in bash.


Feature request: Add arbitrary comments/headers in PDF files to support formats such as PCLm

PCLm is a strict subset of PDF which requires the following format in the first two lines of the file:
%PDF-VERSION_NUMBER
%PCLm-VERSION_NUMBER

For instance, it can be:
%PDF-1.3
%PCLm-1.0

For adding additional headers, QPDFWriter object has a function setExtraHeaderText(std::string const&). When used, the PCLm Magic Number(line 2 in expected output) gets added to the third line of the file rather than the second line. The resulting file looks like:
%PDF-1.4
%BINARY_ENCODING
%PCLm-1.0

There is no such function in QPDF to support extra headers in specific lines. Since PCLm is a strict subset of PDF, it would be great if this feature of adding extra header in an arbitrary line in a PDF file can be introduced.
This issue is currently being faced here: https://github.com/sahilarora535/cups-filters/issues/2

Based on our many conversations offline, I'm inclined to not actually do this since I think we'll just handle PCLm as a separate case. Let me know if you think that's not reasonable. I think it's right though because adding extra comments like this is usually not useful outside the context of some specialized purpose that would require other code changes anyway.

OK, let us go the way as we have settled on with Sahil.

Closing issue as we have determined a different way to satisfy the requirement.


sbt leaks metaspace when reloading an empty project

steps
sbt version: from early 1.0.* up to and including 1.3.0-RC1
$ mkdir /tmp/empty-project
$ cd /tmp/empty-project
$ sbt -no-global -Dsbt.version=1.3.0-RC1 -J-Xmx200m -J-Xms200m -J-XX:+ExitOnOutOfMemoryError
...

reload
200x times

problem
The jvm aborts with out of memory error
expectation
The JVM always returns to the same memory usage it had after starting the first time.
notes
A script to run sbt can be found here: https://gist.github.com/jm-g/c91823d8c4c4045728f63a8d2ba128cd
I have also a fix which I will publish soon. I am creating this issue for discussions.

@jm-g thanks for the report. We've done work post 1.3.0-RC1 to reduce the metaspace utilization of sbt. I'm curious what your fix is. Can you describe the basic idea?

Yes, I've pushed it to https://github.com/jm-g/sbt/tree/fix-4773-memory-leak . This branch is based upon 1.3.0-RC1, I can rebase it on develop this evening.
The root problem imho are left over appender in the global log4j LogManager.  I've added an state exit hook to clear those up. Currently, there are log message in it, that I would change to debug. But I am in a hurry.

The above mentioned branch also contains my attempt to add scripted test for the memory leak. But it needs different scriptedLaunchOpts as all the other tests. Is it possible to set JVM parameters for only one test? Should I increase the number of reload attempts until it also fails with 1.5 GiB heap size? Do you want tests for memory leaks at all?

I'm not surprised that the problem is the log appender. I am curious what happens if you run off of the develop branch because I fixed a log appender related leak after 1.3.0-RC1 in. https://github.com/sbt/sbt/commit/20f6d2243952f09ef9290a4ea7747a8c682165c9.
I think a test is a good idea. My suggestion is that you might try a non-scripted test. You could more or less adapt your shell script approach to a scala test in the sbtProj. You could either reuse the infrastructure in ServerSpec, which spins up an sbt server process and sends commands via sbt's json api or you could just use the java.lang.Process api and write a number of reload commands using the Process outputStream. In either case, you should be able to set the heap parameters when the sbt process is forked.

I've rebased my branch to develop and created a pull  request. But I have skipped the test, because your  (@eatkins)  metaspace fixes make the time to show the memory leak way longer. Now, it takes ca. 10 minutes of continous reloading, without my PR, to reach an oom. With my fix I can reload for 50 minutes without an oom. Those times make it impractical to run automated tests. But I am open to ideas, and I am willing to spend some more time to create a test. Or we could the script from the gist above to the sbt repository as a tool to check for memory leaks manually.
If you look at the debug output during reload, you can see that with time more and more appenders are cleared by my fix. But the amount of classloaders does not grow. Which indicates to me, that there is something accumulating in State which leads to ever more appenders  are being added.


Still not working, plus ad beginning screen shows and flashes white sporadically (literal epilepsy trigger warning btw if anyone else has had that bug  happen)

Script version
What version is the script on eg 2.9 , 3.1 ect
Describe the bug
A clear and concise description of what the bug is.
Expected behavior
A clear and concise description of what you expected to happen.
Screenshots
If applicable, add screenshots or a video to help explain your problem.
Desktop (please complete the following information):

OS: [e.g. iOS]
Browser [e.g. chrome, safari]
Version [e.g. 22]

Additional context
Add any other context about the problem here.

Check out 3.3, This should now be fixed.
(Closed as you did not fill out the issues template)


feature_column.sparse_column_wth_embedding wrong name

The DeepRec docs are open source! To get involved, read the user documents: https://deeprec.readthedocs.io/en/latest/index.html
URL(s) with the issue
Description of issue (what needs changing)

https://deeprec.readthedocs.io/zh/latest/Feature-Eviction.html

good catch, @Tianna-Liu can you submit a fix about the issue?

fixed, https://github.com/DeepRec-AI/DeepRec/pull/844


lab: Add MR and Issue note delete functionality

Add the ability to delete notes from issues and MRs with the --deletenote
option.
Signed-off-by: Prarit Bhargava<EMAIL_ADDRESS>

@prarit Can you add some usage examples to the doc?

Yep, fixed and pushed.

@prarit now the build test broke due to gofmt on the label thing.
But other than that. LGTM.

@prarit now the build test broke due to gofmt on the label thing. But other than that. LGTM.

Grr..  fixed.


Disable concurrent Jenkins builds

Fixes #4, though it's possible I messed up the syntax.

seems like Jenkins doesn't like this

I think you need the options tag to be inside the call to node for it to work perhaps?

The one-liner was having problems, so I'm just gonna go back to the original commit.

Ok, that works too


Wrapping application in router causes  navbar to have 'active' CSS classes applied

I'm playing with the React Router for the first time today. I like it a lot. I just have one odd issue popping up in a sidebar nav:

The first and last
 items in the nav have the CSS active class applied them even though I have not done this in any part of the code. To the extent that I can tell there are no mistakes directly in the CSS - everything is a Bootstrap theme. I haven't customized it yet.
I read in the API docs that an activeClass is not set by default, but yet here I am. Trying to understand if I'm doing something wrong. Doesn't matter which route is really active - the class always sticks.
Here is my router:
    // Create app root element
    <Router history={browserHistory}>
        <Route path="/" component={ApplicationRoot}>
            <IndexRoute component={ViewSearch} />
            <Route path="mysecurity" component={ViewMySecurity} />
        </Route>
    </Router>,

And the HTML for the JSX sidebar component
    render () {
        return(

                            <IndexLink to={`/`}>

                                 Search
                            </IndexLink>

                            <Link to={`/mysecurity`}>

                                 My Security
                            </Link>

                             Logout

        );
    }

Here is the rendered HTML from Safari:
<ul class="nav in" id="side-menu" data-reactid=".<IP_ADDRESS>.0">

<li data-reactid=".<IP_ADDRESS>.0.0">
<a class="active" href="#/" data-reactid=".<IP_ADDRESS>.0.0.0">
<i class="fa fa-search fa-lg" data-reactid=".<IP_ADDRESS>.<IP_ADDRESS>">
<span data-reactid=".<IP_ADDRESS>.<IP_ADDRESS>"> Search

<li data-reactid=".<IP_ADDRESS>.0.1">
<a class="" href="#/mysecurity" data-reactid=".<IP_ADDRESS>.0.1.0">
<i class="fa fa-key fa-lg" data-reactid=".<IP_ADDRESS>.<IP_ADDRESS>">
<span data-reactid=".<IP_ADDRESS>.<IP_ADDRESS>"> My Security

<li data-reactid=".<IP_ADDRESS>.0.2">
<a href="#" class="active" data-reactid=".<IP_ADDRESS>.0.2.0">
<i class="fa fa-lock fa-lg" data-reactid=".<IP_ADDRESS>.<IP_ADDRESS>">
<span data-reactid=".<IP_ADDRESS>.<IP_ADDRESS>"> Logout

Thanks for your question!
We want to make sure that the GitHub issue tracker remains the best place to track bug reports and feature requests that affect the development of React Router.
Questions like yours deserve a purpose-built Q&A forum. Could you post this question to Stack Overflow with the tag #react-router? https://stackoverflow.com/questions/ask?tags=react-router.
We also have an active and helpful React Router community on Reactiflux, which is a great place to get fast help with React Router and with the rest of the React ecosystem. You can join at https://discord.gg/0ZcbPKXt5bYaNQ46.


Enable SSL for Secure HTTPS Access

First off, thanks for the awesome Docker-Raspberry-PI-Monitoring project! It's been super helpful.
Does the project currently support SSL for HTTPS access? I couldn't find any info on setting up SSL certificates and keys to secure the web interface.
If SSL isn't supported yet, could you add it?
Thanks a ton!

I use my Raspberry Pi as a server for multiple projects Im working on and I would like check its resources securely from anywhere. Also from a human human-friendly url
If I find a solution I will share it here

So, you use a domain name? Or a DNS like No-IP?

This issue was closed because it has been open for 30 days with no activity.


[Bug]: Importing the ReRank model through ollama resulted in a keyerror 'ollama'

Is there an existing issue for the same bug?

[X] I have checked the existing issues.

RAGFlow workspace code commit ID
4454b33e(v0.13.0~211) slim
RAGFlow image version
4454b33e(v0.13.0~211) slim
Other environment information
windows 10
ollama version is 0.4.5
model:modelscope.cn/gpustack/bge-reranker-v2-m3-GGUF:latest

Actual behavior
pic
Expected behavior
No response
Steps to reproduce
2024-11-27 12:16:48,084 INFO     19 task_consumer_0 reported heartbeat: {"name": "task_consumer_0", "now": "2024-11-27T12:16:48.079644", "boot_at": "2024-11-27T11:09:40.928496", "pending": 0, "lag": 0, "done": 2, "failed": 2, "current": null}
2024-11-27 12:16:59,379 ERROR    20 'Ollama'
Traceback (most recent call last):
  File "/ragflow/.venv/lib/python3.10/site-packages/flask/app.py", line 880, in full_dispatch_request
    rv = self.dispatch_request()
  File "/ragflow/.venv/lib/python3.10/site-packages/flask/app.py", line 865, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)  # type: ignore[no-any-return]
  File "/ragflow/.venv/lib/python3.10/site-packages/flask_login/utils.py", line 290, in decorated_view
    return current_app.ensure_sync(func)(*args, **kwargs)
  File "/ragflow/api/utils/api_utils.py", line 171, in decorated_function
    return func(*_args, **_kwargs)
  File "/ragflow/api/apps/llm_app.py", line 238, in add_llm
    mdl = RerankModel[factory](
KeyError: 'Ollama'

Additional information
No response

Not supported yet.

Please add support, it's the only thing that prevents using RAGflow fully offline while still preserving decent production-ready RAG results (especially in multi-needle tests).
Awesome app BTW, it's the only one that supports very long documents without context collapse.

olloma does not provide an interface for the rerank model.


Added Cartographic.fromCartesian

For #3177
Added Cartographic.fromCartesian to 3d-tiles. Once this is merged I can also open this pull request into master.

This seems kind of like a weird function to add, since you have to specify the ellipsoid anyway (and the ellipsoid is the thing actually doing the conversion.  Why not just call ellipsoid.cartesianToCartographic directly?

That's what was happening before. It might be useful as a utility function possibly. It would have been nice to let it use a default ellipsoid, but if I try using Ellipsoid.WGS84 I run into cyclical dependencies.

Yes, the circular dependency is exactly why we don't usually have from functions for these types of conversions.  Cartesian3 gets around this by duplicating code, but that's because it's a pretty good win for usability.  I'm not sure that's the case here.

This is the analog of Cartesian.fromDegrees and is a nice usability improvement for picking, which returns a Cartesian, and then displaying the longitude/latitude/height without an explicit ellipsoid.
That reminds me, we should also update this example: https://github.com/AnalyticalGraphicsInc/cesium/blob/master/Apps/Sandcastle/gallery/Picking.html#L177
To allow the WGS84 default and avoid the dependency, we could duplicate the code or, perhaps better, put it in a @private helper function.

+1 for either doing this, or putting in a dummy function or documentation ("looking for .fromCartesian? click here") that points to ellipsoid. cartographic <-> cartesian is a super common operation, but getting a carto from a cart isn't very obvious.

I'm fine with this if we can cleanly factor out a common function (or duplicate a small amount of code) to enable it to default to WGS84

@mramato agreed, I'm confident this can be done cleanly.

I started to look at this more. It looks like there will be a lot of duplicate because of scaleToGeodeticSurface, which cartesianToCartographic calls. Maybe there's a way to condense it, I'm just not familiar with the math. Factoring into an outside function seems strange just to make this work...
Could there be some extra file that contains the WGS84 which Cartographic and Cartesian include?

Could there be some extra file that contains the WGS84 which Cartographic and Cartesian include?

I'm not completely following this.
Why not introduce one new file with a @private scaleToGeodeticSurface function that both Ellipsoid and Cartographic include?  Then just duplicate the fairly trivial implementation of cartesianToCartographic (with a comment) in Carographic.  This code rarely changes so this approach should work well.

Okay I'll do that, I just wasn't sure if scaleToGeodeticSurface was appropriate as it's own file.
What I meant above was have a really simple file called EllispoidWGS84.js that has a single read-only variable which is the WGS84 ellipsoid. That variable can initialized from within Ellipsoid. Then Cartographic and Cartesian can include EllispoidWGS84.js, get the WGS84 ellipsoid, and call the functions that they are currently duplicating.
Or you could do the same thing but put the WGS84 var in Cartographic and Cartesian separately, and initialize them in Ellispoid.

Ah, I see now.  Interesting, but I think this other approach is more straightforward and avoids potential problems by calling these functions during initialization before that WGS84 variable would be defined.

This is ready.

Tweaked the picking example.  Looks good.

This is also ready for a pull request into master.

I updated Cities.html to use it.