Blueprint 404 handlers are not invoked from abort() calls in blueprint views

The docs suggest that, if a blueprint's view function calls abort(404), the blueprint's errorhandler(404) will be called:
https://github.com/pallets/flask/blob/9da947a279d5ed5958609e0b6ec690160c496480/docs/errorhandling.rst#L397-L399
However, that does not seem to be the case:
# sscce.py

from flask import Flask, Blueprint, abort

app = Flask(__name__)
blueprint = Blueprint("", __name__)

@blueprint.route("/")
def router():
    abort(404)

@blueprint.errorhandler(404)
def handler(e):
    return "it worked", 404

if __name__ == "__main__":
    app.register_blueprint(blueprint)
    response = app.test_client().get("/")
    assert response.text == "it worked", response.text

$ python3.8 -m venv venv
$ venv/bin/python -m pip install flask
...
Successfully installed Jinja2-3.1.2 MarkupSafe-2.1.1 Werkzeug-2.2.2 click-8.1.3 flask-2.2.2 importlib-metadata-6.0.0 itsdangerous-2.1.2 zipp-3.11.0
$ venv/bin/python sscce.py
Traceback (most recent call last):
  File "sscce.py", line 22, in <module>
    assert response.text == "it worked", response.text
AssertionError: <!doctype html>

<title>404 Not Found</title>

Not Found

The requested URL was not found on the server. If you entered the URL manually please check your spelling and try again.

This diff causes the handler to be called (but also causes the handler to be called for all views, not just those of the blueprint):
@@ -11,7 +11,7 @@
     abort(404)

-@blueprint.errorhandler(404)
+@blueprint.app_errorhandler(404)
 def handler(e):
     return "it worked", 404

404s are never handled by a blueprint, because at least for a "real" 404 there is no blueprint involved in handling an invalid URL  - the URL did not match a route, and blueprints don't "own" let's say a path prefix (the url prefix of a blueprint is just appended to the routes, multiple blueprints can even have the same prefix).
I assume for consistency this is done regardless of what causes the 404 exception to be raised.
PS: Now I'm curious if raise NotFound has the same behavior...

Yes, raise werkzeug.exceptions.NotFound seems to behave the same way.

Don't give your blueprint an empty name. If you give it a name, like "main", your example works.

Aha! So it does, thank you 👍
Did I miss a memo/doc somewhere about that? Is there a reason Blueprint doesn't raise a ValueError when being instantiated with an empty name?


chore: ci 1.11 to match mix version

Build against the minimum supported version

Reduced the number of builds, not added


Export auth interface to allow external implementations e.g. krb5 - V1-Candidate

The original driver auth interface and implementations have been moved into their own packages and exported. There are two implementations within the driver, NTLM and Windows SSPI.
As before the default on windows is the winsspi package and NTLM on Linux.
The consuming application can now override this at runtime by calling mssql.SetIntegratedAuthenticationProvider(authProvider) e.g.
// create a new auth.Provider around the kerberos client
provider := krb5.NewAuthProvider(krbClient)
// pass the provider to mssql to override the default authentication mechanism
mssql.SetIntegratedAuthenticationProvider(provider)
We have created an integratedauth.Provider implementation here https://github.com/bet365/go-mssqldb-auth-krb5 which uses http://github.com/jcmturner/gokrb5 to authenticate with active directory.
This keep the main driver free from the krb5 dependency and opens the possibility for other implementations in future.
If this pull request is accepted we will update our implementation to the main repo master. Currently it is using a replacement to point at a branch.

Hi David. Sorry for the lateness of this reply, I've been on annual leave.
The 702 PR is quite different from our approach here but I think the base question is "Do you want to take a dependency on github.com/jcmturner/gokrb5 in go-mssqldb itself?"
As you say, ours is reasonably modular, the changes were only just enough to allow external packages to supply the required functionality at runtime, windows was already handled, we added linux. We had in mind a second repo which held the actual reference implementation which users could depend on if they required kerberos on linux. Our example implementation is here https://github.com/bet365/go-mssqldb-auth-krb5 but we dont expect it being the defacto implementation or repo. MS could have it if you like. We could extend it to show other common patterns quite easily, flesh it out.
Have you got a set of requirements in mind for this? Or any comments about the PR as it stands?

Hi everyone, I've been occasionally watching deniskom/#702 and now it seems like this fork is on good way to finally make the krb5-based auth possible.
From someone who simply uses the go-mssqldb module for many use cases, I like the modular approach as well but I think having working krb5 auth in this repo would help many much more, the mentioned PR has quite a lot of people supporting it.
In this particular need to have krb5-based auth, adding only auth interface to allow external implementations could lead to having different implementations in different packages or developers would have their own implementations that could later get outdated or loose maintenance workforce.
I would support both the option to have external auth implementations and also having built-in krb5-based auth that would cover most of the use cases.

thx for the input.
This comment from 702 seems relevant:
https://github.com/denisenkom/go-mssqldb/pull/702#issuecomment-1113844324
We are trying to move the azure identity dependency out of the core module altogether with #30, which makes the azuread package a separate module. That use case is a little different, as the app as to create an AAD-specific connector.
For krb, it seems like it's not a big change for consumers of the driver to import two modules - the core mssql and a krb5 implementation. We'd be happy to host a krb5-based module in this repo, and it could be a version of the code from https://github.com/denisenkom/go-mssqldb/pull/702
With that in mind - what would it take to modify this PR to allow a module to automatically register itself as the krb auth implementation when an app imports it? Is it just a matter of the driver adding an init function that calls mssql.SetIntegratedAuthenticationProvider with an instance of itself?

@shueybubbles can we move ahead and raise a PR cloning changes from https://github.com/denisenkom/go-mssqldb/pull/702 ? or are we looking for some modifications here? Would it be possible to take the changes for the modular approach as an enhancement later on ?

Hi everyone.
We're still working on the more modular approach here that was previously mentioned.
Essentially it gives a default kerberos auth provider, on linux, but allows overriding. We are going to give the auth providers access to the parsed connection string so other implementations could make use of them.
This would still keep krb5 out of the base driver, and would be an additional package used only on linux via an additional import.
One question I would have is if the 702 PR is restricted to just keytabs, or can raw credentials be used?

Hi.
I've pushed a new set of changes for this that add, I believe, everything we wanted.
We keep the krb5 packages separate from the main driver, expose a modular registration approach for authentication, change the interface into authentication to expose the whole connection string
The reference kerberos authentication implementation can be found here, as before.
https://github.com/bet365/go-mssqldb-auth-krb5
It now supports three modes of auth. KeyTabs, Cached Credential Files and Direct Username & Password.
All the values are specified on the connection string, including authentication provider.
I very much look forward to hearing any feedback you might have.

@chandanjainn @stuartpa @PeteBassettBet365 Chandan is trying to implement kerb5 in this repo at #35
I think Chandan's change could be adapted here by converting his additions to an integratedAuth provider. The question is - whose change should go in first? The second person would need to adapt their code to the first.
Chandan is already isolating most of his code in linux-specific files so moving that code around a bit shouldn't be too tricky.

With the modular approach, you can pick any kerberos implementation you like :-) I'll have a look at his in the morning.

Codecov Report

Merging #15 (7fd8e3b) into main (2d408c3) will increase coverage by 0.43%.
The diff coverage is 57.14%.

@@            Coverage Diff             @@
##             main      #15      +/-   ##
==========================================
+ Coverage   72.44%   72.88%   +0.43%
==========================================
  Files          25       24       -1
  Lines        5596     5447     -149
==========================================
- Hits         4054     3970      -84
+ Misses       1302     1236      -66
- Partials      240      241       +1

Impacted Files
Coverage Δ

tds.go
66.62% <37.50%> (-1.23%)
:arrow_down:

mssql.go
85.78% <75.00%> (-0.47%)
:arrow_down:

auth_windows.go
100.00% <100.00%> (ø)

token.go
64.29% <0.00%> (-0.59%)
:arrow_down:

integratedauth/winsspi/winsspi.go

ucs22str_386.go

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

pls address the lint issues.

Hi. I've commit for the linting issues. Is there a problem with the pr-validation job? It seems to fail intermittently. I committed a whitespace change earlier and it flew through, while the previous push failed.  It does it on other prs too.

@shueybubbles any update please? Have you reached a decision which PR would go in first and if the current PR requires any more changes?

I expect we would have to go first. Then a chosen kerberos implementation could slot into that framework.

Some of the tests are timing dependent and can fail intermittently, unfortunately.

@stuartpa I'd appreciate your review, and then I'd merge this one first.
thx all. This experience has been very illuminating for me to issues faced in the field.
In the end I think we'll be setting a pretty good example of driver modularity, in contrast with certain other Microsoft-provided drivers for other languages that pull in a gazillion dependencies.

hi, I should reiterate that we are fine with MS having the reference kerberos provider, which is currently in our repo. Fork it, or similar, and keep it going forward. The import paths for the driver and provider would then be nice and clean.
We'd be fine to continue contributing in either case.

Hi. I've dropped a new commit here with brings our reference kerberos implementation (https://github.com/bet365/go-mssqldb-auth-krb5) into the driver as the integratedauth/krb5 package.
It supports three modes of auth. KeyTabs, Cached Credential Files and Direct Username & Password. All the values are specified on the connection string, including authentication provider. I've updated the read me to detail this.
This does mean that the driver now has a dependency on github.com/jcmturner/gokrb5/v8, meaning it will be fetched regardless of the users need. We can revert this back very simply if that is what you would like. Then it would be a two repo solution again. I feel that this auth provider would be better as it's own repo, so the user only pays for it if they need to use it. We are fine with that being in your org, here on github if you so wish.
Please let me know how you want to play this.

@PeteBassettBet365 @shueybubbles we are good with the modular approach suggested by Pete and will adapt to it, but it's a humble request can we not disregard our PR https://github.com/microsoft/go-mssqldb/pull/35 here ? A lot of effort and time went into the PR since we raised it with the other repo and would like to iterate it came in way before this ?

i think the dependency only has an effect on apps that import the package, and you haven't added the import by default to auth_unix.go.

In reply to:<PHONE_NUMBER>

I appreciate the community enthusiasm behind getting a working kerberos implementation in the driver and ultimately I think everyone will get benefit if can reframe this discussion. Rather than considering #35 as competition with this one, can we look at having 2 PRs as a chance to collaborate on a shared solution both groups will be happy with? I am not an expert on Linux kerberos so I'm not qualified to say whether one implementation proposed is more suitable for inclusion in the core driver than the other. Can this PR evolve as a merger of the two by incorporating components of #35 ? What can I do to facilitate it? Would creating a new branch for interim PRs to merge help?

In reply to:<PHONE_NUMBER>

Reply to https://github.com/microsoft/go-mssqldb/pull/15#issuecomment-1219744980
@shueybubbles yes, definitely. As I previously mentioned we are good with the modular approach suggested by Pete for modularity, all we ask for is to keep the kerberos implementation from https://github.com/microsoft/go-mssqldb/pull/35 and we are happy to adapt to changes according to Pete's PR (once it's merged in) and with any more suggestion the community has for us.

does anyone have docker scripts for setting up a test system with a SQL server image and a Linux container with the kerb client installed to test the kerb auth implementation? I'd like to automate it.

ok, I have reverted that last commit and we are back to having just the authentication provider system in this PR.
A small point but can I ask for everyone's opinions on the Provider interface?
type Provider interface {
	// GetIntegratedAuthenticator is responsible for returning an instance of the required IntegratedAuthenticator interface
	GetIntegratedAuthenticator(config msdsn.Config) (IntegratedAuthenticator, error)
}

In the previous commit I changed it to take a pointer to msdsn.Config as the config is growing. It wont make a massive difference as these functions dont get called very often, but it would make some difference. The other side of that coin is that the providers shouldn't really be able to make changes to the config and as such passing by value is right. Just a thought. @chandanjainn you and I need to agree on this :-)
If no further changes are suggested I think we can move ahead with this PR.
On the second point of an actual krb5 implementation, I have no problem working with @chandanjainn toward that. It was not my intention to marginalise your effort. We all want to see the driver evolve in the best way possible which is why we got involved after having to implement the kerberos auth ourselves internally. We've been running a version of this auth for 18 months and in production for over a year. It now sits behind a very substantial, and growing, percentage of our business. I'm sure we can ensure the final version meets all our needs.
Thirdly @shueybubbles In reference to https://github.com/microsoft/go-mssqldb/pull/15#issuecomment-1219727343 I believe any package referenced in go-mssqldb go.mod will be downloaded onto all developers machines, including all their deps etc. You are right that not importing the krb5 packages on windows will cause them to not form part of the output binary, but the source code will still be on their disk. That's why I was suggesting a second repo for these. The krb5 packages would not be part of the go-mssqldb go.mod file at all and it would then be the developers choice to add both the repos for go-mssqldb driver and the krb5 packages to the go.mod file of their client application. If dev disk space is not a concern (it'll likely only be a few mb) then the krb5 auth package can be part of this repo.

thx all
I think passing the config by value provides the right interface and clear expectations that the config is immutable. I suppose an alternative would be to define a config interface so you could pass the pointer but protect the data by only using the interface.
As for disk space consumption from unused dependencies, I think that ship sailed when I added the azuread support. Also,  a fair number of folks like having a default implementation in the same repo as the driver where it's easy to find.
If everyone agrees on the pass by value config we can approve and merge this, then Chandan can adapt his PR to it.

In reply to:<PHONE_NUMBER>

Reply to<PHONE_NUMBER> and<PHONE_NUMBER>
@PeteBassettBet365 @shueybubbles I agree with passing by value for the config. Even I had switched pass by value from pass by reference in my recent commits considering this.
@shueybubbles are we looking for a script just for Krb client or the KDC server too?

Preferably a script for both, since kerb auth doesn't need a full Windows domain (afaik). Given such a script I can integrate it into the Github action to run driver tests, and I'll reuse it for go-sqlcmd testing.

In reply to:<PHONE_NUMBER>


Update findXmasSunday function arguments to (1970, 2017)

Update findXmasSunday function arguments from (2008, 2121) to (1970, 2017) to match expected return.
Checklist:

[x] I have read freeCodeCamp's contribution guidelines.
[x] My pull request has a descriptive title (not a vague title like Update index.md)
[x] My pull request targets the master branch of freeCodeCamp.
[x] All the files I changed are in the same world language, for example: only English changes, or only Chinese changes, etc.

Closes #39995

congratulation for your first merged pull request! 🥳
if you want to contribute more, check the issues labelled "help wanted"!


md-select panelClass does not apply classes

Bug, feature request, or proposal:
Per the md-select documentation the panelClass input should apply the specified classes to the select panel and function the same as ngClass. Specifying a class in the input doesn't apply the desired class.
What is the expected behavior?
Classes that are indicated in the panelClass input should be applied.
What is the current behavior?
The class specified is not applied to the element.
What are the steps to reproduce?
Plunker
What is the use-case or motivation for changing an existing behavior?
In my specific use case I am using a md-select on a dark background and wish to change the color of the selected value that's being displayed to something lighter.
Which versions of Angular, Material, OS, TypeScript, browsers are affected?
Material 2.0.0-beta.8
Typescript 2.1.6
Angular 4.1.3
Is there anything else we should know?
Thanks for the help!

Take a look at @willshowell's https://github.com/angular/material2/issues/5450#issuecomment-312712854


[Bug] | Plex Home users don't work

Are you using the latest release?

[X] Yes, I am using the latest release.

Have you checked a bug report doesn't already exist?

[X] Yes, I have checked the existing bug reports.

What happened?
When adding a plex home user using the invite option, it doesn't work and gives the error:
"We were unable to accept the plex inite on your behalf, please accept the invite manually through plex or your email". But there is no invite on the plex account or email.
On the plex owners account a new user has been added to home as a managed account with just an email address and not the invited users plex account.

Virtualisation
Docker
What operating system are you seeing the problem on?
Linux
What browsers are you seeing the problem on?
Chrome
Project Location
Local
Bug Description
When trying to add a plex home user, it does not work and simply adds the plex accounts email to the home as a managed account.
Steps to Reproduce
When creating a invite for plex, select "Allow Plex Home Access" and try to sign up with a plex account
Relevant Log Output
No response

This feature does not work and cannot work. I was investigating this yesterday and the API endpoints just don’t exist for it unfortunately.
So, this feature is being removed within #407 and cannot be fixed unfortunately.
Plex does now allow the invitation of existing Plex users through their API, only the creation of Managed Users.

Ok, not too difficult to add them to plex home manually once they have been invited, thanks for letting me know


Deleting teams doesn't work

I created a test team, invited my gmail, then tried to delete the team. Not sure if it's relevant that there was a pending invite. Anyway, here's the recursion error:
https://dimagi.sentry.io/issues/4806050600/?alert_rule_id=14063560&alert_timestamp=1704364280155&environment=production&notification_uuid=f5834af6-e852-454b-91f3-577757112f20&project=4505001320316928

This is related to the audit library we're using. I get a "maximum recursion depth exceeded" when trying to delete an audited model. I've only found this happening with deletions, and only on models that have some sort of relationship with another one, but I suspect there must be multiple relationships to trigger this. We can expect the models ticked on this list to have this behaviour. My gut feeling says it should be fine to leave this as-is for now, since logging is more important than deleting stuff, but I'm happy to temporarily remove auditing if need be @bderenzi.

Reproduced the error in the audit library here: https://github.com/dimagi/django-field-audit/compare/cs/max_recursion_error?expand=1

This is fixed


[QnA Maker] Declarative tests for QnA Maker dialog and some bug fixes

Declarative tests for QnA Maker dialog and some bug fixes

:heavy_check_mark: No Binary Compatibility issues for Microsoft.Bot.Builder.dll compared against version 4.5.3
:heavy_check_mark: No Binary Compatibility issues for Microsoft.Bot.Builder.AI.Luis.dll compared against version 4.5.3
:heavy_check_mark: No Binary Compatibility issues for Microsoft.Bot.Builder.AI.QnA.dll compared against version 4.5.3
:heavy_check_mark: No Binary Compatibility issues for Microsoft.Bot.Builder.ApplicationInsights.dll compared against version 4.5.3
:heavy_check_mark: No Binary Compatibility issues for Microsoft.Bot.Builder.Azure.dll compared against version 4.5.3
:heavy_check_mark: No Binary Compatibility issues for Microsoft.Bot.Builder.Dialogs.dll compared against version 4.5.3
:heavy_check_mark: No Binary Compatibility issues for Microsoft.Bot.Builder.TemplateManager.dll compared against version 4.5.3
:heavy_check_mark: No Binary Compatibility issues for Microsoft.Bot.Configuration.dll compared against version 4.5.3
:heavy_check_mark: No Binary Compatibility issues for Microsoft.Bot.Connector.dll compared against version 4.5.3
:heavy_check_mark: No Binary Compatibility issues for Microsoft.Bot.Schema.dll compared against version 4.5.3

You should not need to add the package <PackageReference Include="RichardSzalay.MockHttp" Version="5.0.0" />

:heavy_check_mark: No Binary Compatibility issues for Microsoft.Bot.Builder.dll compared against version 4.5.3
:heavy_check_mark: No Binary Compatibility issues for Microsoft.Bot.Builder.AI.Luis.dll compared against version 4.5.3
:heavy_check_mark: No Binary Compatibility issues for Microsoft.Bot.Builder.AI.QnA.dll compared against version 4.5.3
:heavy_check_mark: No Binary Compatibility issues for Microsoft.Bot.Builder.ApplicationInsights.dll compared against version 4.5.3
:heavy_check_mark: No Binary Compatibility issues for Microsoft.Bot.Builder.Azure.dll compared against version 4.5.3
:heavy_check_mark: No Binary Compatibility issues for Microsoft.Bot.Builder.Dialogs.dll compared against version 4.5.3
:heavy_check_mark: No Binary Compatibility issues for Microsoft.Bot.Builder.TemplateManager.dll compared against version 4.5.3
:heavy_check_mark: No Binary Compatibility issues for Microsoft.Bot.Configuration.dll compared against version 4.5.3
:heavy_check_mark: No Binary Compatibility issues for Microsoft.Bot.Connector.dll compared against version 4.5.3
:heavy_check_mark: No Binary Compatibility issues for Microsoft.Bot.Schema.dll compared against version 4.5.3


Reprocess Streams not working

Stroom v7 UI problem: Although asked to confirm reprocess, the processor filter is not added to the pipeline.

Fixed.


Add more demos to the hooman/demos folder

Hi, I have created a brick breaker game using hooman library. Please verify it.

I have made the necessary changes. Can you please verify it.


Non-interactive model download

This PR adds the following improvements:
Support for the HUGGINGFACE_TOKEN environment variable
Ensures that the SD weights download proceeds in a truly non-interactive fashion when the --yes flag is used and a valid authentication token is present in any of the supported locations (env vars or cached).
Tested as follows:

HUGGING_FACE_HUB_TOKEN env var is not empty:

with --yes: weights download proceeds; token is not persisted in cache
without --yes: weights download proceeds without an additional prompt; token is not persisted in cache

HUGGINGFACE_TOKEN env var is not empty:

same behaviour as with the official var (token not persisted)

Env vars not present without --yes flag: user is prompted for the token, token is cached only if valid (same as current behaviour)
Env vars not present with --yes flag: weights download is skipped (same as current behaviour)
In any case where the token is invalid (i.e. login to HF does not succeed), the token will not be cached.

Additionally:

fixes a bug where outdir location was not printed due to missing f-string
fixes spelling of --outdir argument in a message
implements a way of exposing download failures at the end of configuration run. if model download failed or was skipped, the postscript message was misleadingly suggesting that the application was ready to use.

huggingface_hub.login(token) could be a more-explicit alternative to resetting os.environ inside the script.
Also note that they've changed the settings on some of the repos so now not all of the Stable Diffusion models require a token. As of last time I checked earlier this weekend, the CompVis/ ones don't (SD 1.4) but the runwayml/ ones do (SD 1.5 + SD 1.5 inpainting).

huggingface_hub.login(token) could be a more-explicit alternative to resetting os.environ inside the script.

That is how I initially implemented it, but the behaviour becomes inconsistent between using the officially supported env var HUGGINGFACE_HUB_TOKEN. and the de-facto "community standard" HUGGINGFACE_TOKEN. Calling the .login() method causes the token to be saved in ~/.huggingface/token, whereas using the official env. var does not.

Moving the initialization file is proving tricky if we want to maintain backwards compatibility with the current setup, because we want to move it into the runtime dir, but users might already have a conflicting --root-dir argument  in an existing ~/.invokeai file. Also, setting it from Globals before checking for a user-specified runtime dir creates a catch-22.
TLDR; Perhaps the initfile should be dealt with in a separate PR so that this one isn't blocked for too long. Unless we decide to break backwards-compatibility, then it's easy

Moving the initialization file is proving tricky if we want to maintain backwards compatibility with the current setup, because we want to move it into the runtime dir, but users might already have a conflicting --root-dir argument in an existing ~/.invokeai file. Also, setting it from Globals before checking for a user-specified runtime dir creates a catch-22.
TLDR; Perhaps the initfile should be dealt with in a separate PR so that this one isn't blocked for too long. Unless we decide to break backwards-compatibility, then it's easy

Why not have an environment variable that contains the path to the init file?
As an aside, introducing Globals wasn't my favorite approach, but it was better than carrying a directory path across multiple nested functions.
Just to set expectations, unless the release gets delayed significantly I'm going to go with the existing configuration script for 2.2.0, and release the new and improved one that you're working on for 2.2.1. This is primarily because I've got a lot of documentation-writing to do and won't be able to do thorough testing on this. I very much appreciate your work on this bit of the code.
I also have a suggestion for a new feature. Either as a command-line option or as an automatic feature during interactive processing, it would be great if the script could scan the existing models directory for SD models it knows about (from INITIAL_MODELS.yaml). If it finds SD models that aren't in the config file, it should offer to add them. This will let people rebuild their config.yaml without having to ask the script to download the models and answer 'y' to each prompt.

That being said, if I do have time I will try very hard to get this in. The improvements look very good.

All sounds great to me. I actually might have a viable approach to handling the .invokeai config file location in a backwards-compatible manner, but certainly wouldn't want any of this to delay the release. If anything, that would give me more time to do some more refactoring and properly test the changes. But I'm aiming try to get the final fixes in tonight.

I think I was able to wrangle the config file location into a good state; this is now ready for review.
@lstein would you like me to tackle the auto-scanning of models in this same PR, or open a new feature request for it? I'd prefer the latter as it might get messy to review otherwise, but it's your call, please let me know.

I think I was able to wrangle the config file location into a good state; this is now ready for review.
@lstein would you like me to tackle the auto-scanning of models in this same PR, or open a new feature request for it? I'd prefer the latter as it might get messy to review otherwise, but it's your call, please let me know.

Go ahead and start a new PR. I'm going to integrate what you've done here with a few minor changes I made on a different branch and then merge.

I seem to have oddly broken some tests, let me look into that :thinking:

I started testing and then realized that the default location for the init file has become the InvokeAI repo.

That sounds unexpected and not intentional. The logic I was trying to implement was to: 1. if an existing config file is found in the user's home directory, then keep using it, and 2. if no existing file is found, then create one in the runtime directory.
If the default location of the runtime directory happens to be with the root of the repo, then yes, this is a problem. I think this stems from:
# This is usually overwritten by the command line and/or environment variables
- Globals.root = '.'
+ Globals.root = root_dir if (root_dir := os.getenv("INVOKEAI_ROOT")) else "."

But this behaviour should have remained unchanged if neither the env var nor the --root_dir are specified. I.e. Globals.root was already "." before this PR.
As far as I understand, we never want to mix the runtime dir and neither the cloned codebase nor especially the dev-installed Python module, as you mention. Would it make sense to instead default the runtime dir to be located at ~/invokeai/ instead of the more ambiguous "."?

Yeah, I think we need to reverse the logic and have a predictable location of the invokeai directory rather than a predictable init file. The init file can then live inside invokeai.

Just marking this as draft again while we work out the strategy.

I also just noticed that the configure_invokeai.py, the initfile comments, and the CI tests all specify the --root option, while in args.py, CLI.py and elsewhere in the app we have --root_dir.
@lstein perhaps leaving this PR unmerged until after 2.2.0 is indeed for the best, so that we can clean it all up after the release dust settles. I will keep rebasing+updating it though. Do you agree?
(CLI argument handling and various entrypoints could use quite a bit of future refactoring + consolidation in general, IMHO)

I moved the init file / root dir changes to another branch and will do a separate PR for it, as it was getting quite messy and needs a larger refactor.

This is ready to be reviewed again, because the runtime dir / init file location refactoring will be dealt with separately, likely in https://github.com/invoke-ai/InvokeAI/pull/1615

Very useful for installation via UI 🍾


Unexpected token export

Using the module as described in the readme (installed with npm), I get this error:
> ts-node src/gpt.ts
[somewhere]/node_modules/offset-polygon/lib/offset-polygon.js:165
export {
^^^^^^

SyntaxError: Unexpected token 'export'
    at internalCompileFunction (node:internal/vm:73:18)
    at wrapSafe (node:internal/modules/cjs/loader:1153:20)
    at Module._compile (node:internal/modules/cjs/loader:1197:27)
    at Module._extensions..js (node:internal/modules/cjs/loader:1287:10)
    at Object.require.extensions.<computed> [as .js] (/usr/lib/node_modules/ts-node/src/index.ts:1608:43)
    at Module.load (node:internal/modules/cjs/loader:1091:32)
    at Function.Module._load (node:internal/modules/cjs/loader:938:12)
    at Module.require (node:internal/modules/cjs/loader:1115:19)
    at require (node:internal/modules/helpers:130:18)
    at Object.<anonymous> ([somewhere]:17:1)

however, copy/pasting the code of offset-polygon.ts to a local file and importing that, works fine.
any idea what's going on?

Hey,
export is a part of ESM module specification, so I guess you probably need to enable ESM support in ts-node:
https://www.npmjs.com/package/ts-node#esm
Let me know if that works.
Cheers!


When going to Word History the data is not up to date

The data displayed in Word History Tab is not accurately reflecting the state of the database. Namely, if a user translates an article and then goes to the page the data won't be present there.

I haven't really been able to reproduce this, so it's likely not due to time, but maybe number of transactions that causes the reads to go stale?
Some of the possible solutions suggested closing the Sessions after they are used, so maybe we can try to see if that makes a difference in this case.

This should be even easier to test then. Just hit the test DB with a lot of read and write requests and see whether after a certain number they go stale!

In fact, creating a synthetic workload that we can hit the API with might be cool also for performance testing -- seeing how the API scales when we hit it with 1000 connections at the same time, etc.

Sure, I think that sounds like a good idea - but how would you simulate this - would it be a front end script that makes a bunch of API async calls?
Currently, I am inserting all the bookmarks in the DB for 2 test users and see if after that I have any issues.

I have tried inserting 200k Bookmarks and spamming the API from the Frontend inserting values in the user activity table, but still the history seems OK to me. I am a bit unsure what else I could try, if anything I think this was more intense than what our server is doing.
Is there any other setting that sets Flask to "production" or something  that might not be activated in the dev environment?

Sure, I think that sounds like a good idea - but how would you simulate this - would it be a front end script that makes a bunch of API async calls?

yes. a script that hits the API endpoints in some kind of meaningful scenario replicating user behavior. i was thinking more as python than as frontend.

Currently, I am inserting all the bookmarks in the DB for 2 test users and see if after that I have any issues.

I have tried inserting 200k Bookmarks and spamming the API from the Frontend inserting values in the user activity table, but still the history seems OK to me. I am a bit unsure what else I could try, if anything I think this was more intense than what our server is doing.

how did you insert the 200k bookmarks?
i think we should really try so simulate a client sending api requests.
my suspicion is that it has to go through the API for the problem to be revealed.
we are using this Flask-SQLAlchemy extra layer that does it's own tricks with transactions, that i never managed to look attentively into. Problem might be there.

Is there any other setting that sets Flask to "production" or something that might not be activated in the dev environment?

not that i'm aware of.
this is the only config param that i'm seeing that might be related
Required by API
DEBUG=False

I will also say that I can easily see this behaviour in the API - just by translating a word, it never comes up in the history - so one would think it wouldn't be as difficult to reproduce!

So again, a python app that does the following:

login a user
get a translation
wait a minute
get the history
check that the translation is in there
until this last condition does not hold would be the simplest test script then

Alternatively, maybe the problem is only in production and not in debug.so try to change your config locally to debug false?

I can set up the python application to work as a client, but the translations will just be a random string (as I don't have API access to google, and it wouldn't make sense to use our budget in this situations.
I will try with the debug=FALSE

YES. Random strings are fine!
On 5 Sep 2024 at 16.05 +0200, Tiago Ribeiro @.***>, wrote:

I can set up the python application to work as a client, but the translations will just be a random string (as I don't have API access to google, and it wouldn't make sense to use our budget in this situations.
I will try with the debug=FALSE
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you commented.Message ID: @.***>

Okay - so I managed to get it to fail - not while running, but essentially I wrote the script and let it run and hibernated the computer when closing off yesterday.
Today, when re-logging the issue is present - which makes me thing, could it be related with the session idling in some way?
I will have the script running this morning with 3 different accounts and see if it occurs under normal use, but this is at least something!

Yes. That's the beginning of the end for this bug! :)

I have the feeling that we will have to understand this discussion because it might be relevant for us.

I see - I am not sure I fully get it, as I haven't set up the Session and Model, but I also notice that after opening a connection it doesn't seem it gets closed up until the user signs out - by looking at the process lists, it can quickly grow with all the connections being left on sleep:

I also try to replicate it by just hibernating and resuming the computer but that isn't enough so the time it's in Hibernation did also have an impact.

those are the connections to the DB? how do you show them?

Run the following query: show full processlist;

alternative hypothesis is that it might block if we don't call the API for a long period, e.g. 2h?
in the sense of, maybe if we call it every minute it never goes stale?
but still, i think it's worth calling it many times first, to see whether that results in the stale data.

I haven't been able to get it to get stale date - with about 1000 calls.
Also update, hibernating for 30ish minutes did not cause the error. So it might be a long timeout associated - Though this wouldn't really explain why a new connection would come in and fail to retrieve the data. So maybe it might be after a certain number of transaction ?

Last week, I left once again 3 scripts adding bookmarks and checking they are retrieved in history, but this time only 2 of them failed, while one is still running.
I guess so far the following as been observed:

The number of insertions doesn't seem to be the main reason. The last time it failed it was with around 800 insertions and now it was with about 900 (491 in each process that failed)
It only seems to happen after I hibernate for a long time, but today it seems that is not the only condition (as one of the processes is still running OK. I checked MySQL timeouts and they seem to be around 8H, this would match with the first time it occurred.
After the error occurs, then I can no longer see the bookmarks of newly translated words, in the frontend - which mirrors what we see in Production. Restarting the SQL seems to fix it as it does in Production.


Dependency Injection Lifespan and transfer of Ownership

Modify I2C interface consumption to transfer ownership to driver and guarantee that the lifespan of the dependency matches that of the HDC1080 driver class.

Determined this is not possible without dynamic allocation.


RRateLimiter分布式限流BUG: 不会删除ZSET数据结构的key。

Expected behavior
无论是服务重启，还是重新加载限流配置（rateLimiter.setRate），都不会删除下列两种ZSET结构的key。
我知道这个key是用于限流统计的。permits用于存放请求记录包括时间戳，value用于统计对应剩余的令牌桶。
KEY:
{xxx}:permits
{xxx}:value
这样会导致一个问题，我如果加载了新的限流配置，比如一秒生成200个令牌（原先是100）；
而上述两种key不会删除或者更新，这样当新的请求进来时，还是从前面的 key:  {xxx}:value 中拿剩余的令牌；
这样就导致，明明新设置了200限流，但是还是被限流100。
Actual behavior
Steps to reproduce or test case
Redis version

org.redisson
redisson-spring-boot-starter
3.22.0

Redisson version
Redisson configuration

Can you share the code to reproduce it?

ok.
1、When the project starts, I will load current limiting configurations for each interface.
rateLimiter.setRate(2, 100, 1, RateIntervalUnit.SECONDS);
2、In the slice, for each interface, attempt to obtain tokens from the token bucket.
final String ofRateLimiter = RedisKeyConstants + key;
RRateLimiter rateLimiter = redisson.getRateLimiter(ofRateLimiter);
if (!rateLimiter.tryAcquire()) {
return Response.createError("接口访问超出频率限制，请稍后重试");
}
3、after, I modified the current limiting configuration of the interface and then reloaded and executed     rateLimiter.setRate(2, 200, 1, RateIntervalUnit.SECONDS);
4、Then, in the slice, when each interface requests, it attempts to obtain available tokens, but at this point, the remaining tokens in the token bucket are the previously configured 100 instead of the latest configured 200
’rateLimiter.tryAcquire()‘
5、That is to say, the latest restriction configuration is not effective. I checked the reason and found that it is because the Redis key that manages the remaining tokens was not deleted, resulting in the latest request still obtaining tokens from the old Redis key.
Redis key :
{xxx}:permits
{xxx}:value       这个key中的值是之前配置的100，而不是最新的200。

@victoyzz
I have attempted to reproduce the issue mentioned using the code available at https://github.com/wynn5a/redisson-rate-limiter-bug, but I was unable to reproduce the problem.
the method org.redisson.RedissonRateLimiter#setRateAsync specifies that {xxx}:permits and {xxx}:value will be deleted upon calling setRate.
Please inform me if there is anything I might have overlooked.

你可以通过  RateLimiterConfig rateLimiterConfig = rateLimiter.getConfig();  判断限流策略是否存在更新。如果更新了，先通过 rateLimiter.delete(); 删除旧限流策略，并配置新的限流策略。

I had the same problem。
In the following code, when I request the interface once per second, the first 5 times are ok, but the sixth time, it will be blocked for more than 10 seconds
rRateLimiter = redissonClient.getRateLimiter("myTestLimiter");
rRateLimiter.trySetRate(RateType.OVERALL, 5, 1, RateIntervalUnit.SECONDS);
@GetMapping("/myTestLimiter")
public Result test() {
rRateLimiter.acquire(1);
return RespResult.success();
}

I had the same problem。
In the following code, when I request the interface once per second, the first 5 times are ok, but the sixth time, it will be blocked for more than 10 seconds
rRateLimiter = redissonClient.getRateLimiter("myTestLimiter"); rRateLimiter.trySetRate(RateType.OVERALL, 5, 1, RateIntervalUnit.SECONDS); @GetMapping("/myTestLimiter") public Result test() { rRateLimiter.acquire(1); return RespResult.success(); }

I am unable to replicate the issue you mentioned. For further investigation, please refer to the following GitHub repository:
https://github.com/wynn5a/redisson-rate-limiter-bug/tree/spring
I conducted tests using a local Redis instance, it is possible that I overlooked certain details.

来信已经收到，谢谢。

@victoyzz
I have attempted to reproduce the issue mentioned using the code available at https://github.com/wynn5a/redisson-rate-limiter-bug, but I was unable to reproduce the problem.
the method org.redisson.RedissonRateLimiter#setRateAsync specifies that {xxx}:permits and {xxx}:value will be deleted upon calling setRate.
Please inform me if there is anything I might have overlooked.

Is the version number consistent?
redisson-spring-boot-starter
3.22.0

As shown in the figure, after retrySetRate, these keys remain in Redis and will not be actively deleted.

I have conducted further testing using Spring Boot Starter, however, I am still unable to replicate the reported issue. Kindly examine my test project on GitHub https://github.com/wynn5a/redisson-rate-limiter-bug/tree/spring-boot-starter  for a thorough evaluation.
Perhaps an unknown Redisson configuration is causing this issue. To identify the root cause, I suggest resetting the configuration to its default settings for investigation.

@wynn5a
Below is the reproducer.
        RRateLimiter rateLimiter = redisson.getRateLimiter("test1");
        rateLimiter.setRate(RateType.PER_CLIENT, 5, 5, RateIntervalUnit.SECONDS);

        for (int i = 0; i < 5; i++) {
            rateLimiter.acquire();
            System.out.println("time1 " + Instant.now());
        }

        Thread.sleep(5000);

        rateLimiter.setRate(RateType.PER_CLIENT, 10, 5, RateIntervalUnit.SECONDS);

        System.out.println("---");
        Assertions.assertTimeout(Duration.ofSeconds(1), () -> {
            for (int i = 0; i < 10; i++) {
                rateLimiter.acquire();
                System.out.println("time2 " + Instant.now());
            }
        });

Yes, the PRE_CLIENT configuration seems to be the key. I will try to address this.

Merged


Unhandled error when close sent after open

If a misbehaved client sends close directly after open, the server will log an unhandled error, sqlite3.IntegrityError.
	Traceback (most recent call last):
	  File "/app/env/local/lib/python2.7/site-packages/twisted/python/log.py", line 103, in callWithLogger
	    return callWithContext({"system": lp}, func, *args, **kw)
	  File "/app/env/local/lib/python2.7/site-packages/twisted/python/log.py", line 86, in callWithContext
	    return context.call({ILogContext: newCtx}, func, *args, **kw)
	  File "/app/env/local/lib/python2.7/site-packages/twisted/python/context.py", line 122, in callWithContext
	    return self.currentContext().callWithContext(ctx, func, *args, **kw)
	  File "/app/env/local/lib/python2.7/site-packages/twisted/python/context.py", line 85, in callWithContext
	    return func(*args,**kw)
	--- <exception caught here> ---
	  File "/app/env/local/lib/python2.7/site-packages/twisted/internet/posixbase.py", line 614, in _doReadOrWrite
	    why = selectable.doRead()
	  File "/app/env/local/lib/python2.7/site-packages/twisted/internet/tcp.py", line 205, in doRead
	    return self._dataReceived(data)
	  File "/app/env/local/lib/python2.7/site-packages/twisted/internet/tcp.py", line 211, in _dataReceived
	    rval = self.protocol.dataReceived(data)
	  File "/app/env/local/lib/python2.7/site-packages/autobahn/twisted/websocket.py", line 131, in dataReceived
	    self._dataReceived(data)
	  File "/app/env/local/lib/python2.7/site-packages/autobahn/websocket/protocol.py", line 1175, in _dataReceived
	    self.consumeData()
	  File "/app/env/local/lib/python2.7/site-packages/autobahn/websocket/protocol.py", line 1187, in consumeData
	    while self.processData() and self.state != WebSocketProtocol.STATE_CLOSED:
	  File "/app/env/local/lib/python2.7/site-packages/autobahn/websocket/protocol.py", line 1553, in processData
	    fr = self.onFrameEnd()
	  File "/app/env/local/lib/python2.7/site-packages/autobahn/websocket/protocol.py", line 1674, in onFrameEnd
	    self._onMessageEnd()
	  File "/app/env/local/lib/python2.7/site-packages/autobahn/twisted/websocket.py", line 159, in _onMessageEnd
	    self.onMessageEnd()
	  File "/app/env/local/lib/python2.7/site-packages/autobahn/websocket/protocol.py", line 627, in onMessageEnd
	    self._onMessage(payload, self.message_is_binary)
	  File "/app/env/local/lib/python2.7/site-packages/autobahn/twisted/websocket.py", line 162, in _onMessage
	    self.onMessage(payload, isBinary)
	  File "/app/env/local/lib/python2.7/site-packages/wormhole/server/rendezvous_websocket.py", line 153, in onMessage
	    return self.handle_close(msg, server_rx)
	  File "/app/env/local/lib/python2.7/site-packages/wormhole/server/rendezvous_websocket.py", line 283, in handle_close
	    self._mailbox.close(self._side, msg.get("mood"), server_rx)
	  File "/app/env/local/lib/python2.7/site-packages/wormhole/server/rendezvous.py", line 145, in close
	    db.execute("DELETE FROM `mailboxes` WHERE `id`=?", (self._mailbox_id,))
	sqlite3.IntegrityError: FOREIGN KEY constraint failed

The preceding log lines are:
2017-10-31T17:49:37+0000 [_GenericHTTPChannelProtocol,39,<IP_ADDRESS>] ws client connecting: tcp4:<IP_ADDRESS>:57522
2017-10-31T17:49:37+0000 [_GenericHTTPChannelProtocol,39,<IP_ADDRESS>] creating nameplate#6 for app_id jml.io/hocus-pocus
2017-10-31T17:49:37+0000 [_GenericHTTPChannelProtocol,39,<IP_ADDRESS>] spawning #whaj7i57fonp4 for app_id jml.io/hocus-pocus
2017-10-31T17:49:37+0000 [_GenericHTTPChannelProtocol,39,<IP_ADDRESS>] Unhandled Error

The client is definitely misusing the server, but this really shouldn't result in a random traceback.
The error occurs even when there's a 5s delay between open and close.

Oh, oops, yeah that shouldn't cause an error.
What were the exact preceding messages delivered to the server? Looks like there was an allocate before the open? I'll see if I can write a test case.

Stuff between >>> lines is outbound from server. Stuff between <<< lines is inbound from client.


Please help debugging external memory race condition.

Hi, @RAMitchell @hcho3 @CodingCat
I encountered an error on master branch a30176907f5628020f8238464c2f58907ee724d1 by running Python test:
pytest -s --fulltrace --verbose ./tests/python/test_updaters.py -k colmaker

The error is not deterministic, sometimes it's a segfault, while other times it's some failing checks,  I attached a few of them at the end.  Also note that if I limit XGBoost to single thread all tests run smoothly.  So I believe that it's a race error.  I ran tsan but pytest got stuck at collecting tests, asan is not really helpful for debugging producer/consumer thread model.  Could you provide some suggestions? I can't think of a place to starting looking...
[02:09:12] WARNING: /home/fis/Workspace/xgb/xgboost/src/gbm/gbtree.cc:95: DANGER AHEAD: You have manually specified `updater` parameter. The `tree_method` parameter will be ignored. Incorrect sequence of updaters will produce undefined behavior. For common uses, we recommend using `tree_method` parameter instead.
[02:09:12] WARNING: /home/fis/Workspace/xgb/xgboost/src/gbm/gbtree.cc:95: DANGER AHEAD: You have manually specified `updater` parameter. The `tree_method` parameter will be ignored. Incorrect sequence of updaters will produce undefined behavior. For common uses, we recommend using `tree_method` parameter instead.
[02:09:12] SparsePage::Writer Finished writing to tmptmp_.sorted.col.page
[02:09:12] SparsePageSource: Finished writing to tmptmp_
[02:09:12] SparsePage::Writer Finished writing to tmptmp_.col.page
terminate called after throwing an instance of 'dmlc::Error'
  what():  [02:09:12] /home/fis/Workspace/xgb/xgboost/dmlc-core/include/dmlc/threadediter.h:189: Check failed: !producer_sig_processed_:
Stack trace:
  [bt] (0) /home/fis/Workspace/xgb/lib/python3.6/site-packages/xgboost-0.90-py3.6.egg/xgboost/./lib/libxgboost.so(dmlc::LogMessageFatal::~LogMessageFatal()+0x4a) [0x7f9ed8329030]
  [bt] (1) /home/fis/Workspace/xgb/lib/python3.6/site-packages/xgboost-0.90-py3.6.egg/xgboost/./lib/libxgboost.so(dmlc::ThreadedIter<xgboost::SparsePage>::BeforeFirst()+0x126) [0x7f9ed8385830]
  [bt] (2) /home/fis/Workspace/xgb/lib/python3.6/site-packages/xgboost-0.90-py3.6.egg/xgboost/./lib/libxgboost.so(xgboost::data::SparsePageSource::BeforeFirst()+0xa9) [0x7f9ed8381269]
  [bt] (3) /home/fis/Workspace/xgb/lib/python3.6/site-packages/xgboost-0.90-py3.6.egg/xgboost/./lib/libxgboost.so(xgboost::data::SparsePageDMatrix::GetRowBatches()+0x74) [0x7f9ed837b020]
  [bt] (4) /home/fis/Workspace/xgb/lib/python3.6/site-packages/xgboost-0.90-py3.6.egg/xgboost/./lib/libxgboost.so(xgboost::data::SparsePageSource::CreatePageFromDMatrix(xgboost::DMatrix*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, unsigned long)+0x35b) [0x7f9ed8382781]
  [bt] (5) /home/fis/Workspace/xgb/lib/python3.6/site-packages/xgboost-0.90-py3.6.egg/xgboost/./lib/libxgboost.so(xgboost::data::SparsePageSource::CreateColumnPage(xgboost::DMatrix*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool)+0x7f) [0x7f9ed8383033]
  [bt] (6) /home/fis/Workspace/xgb/lib/python3.6/site-packages/xgboost-0.90-py3.6.egg/xgboost/./lib/libxgboost.so(xgboost::data::SparsePageDMatrix::GetColumnBatches()+0x5b) [0x7f9ed837b38d]
  [bt] (7) /home/fis/Workspace/xgb/lib/python3.6/site-packages/xgboost-0.90-py3.6.egg/xgboost/./lib/libxgboost.so(xgboost::data::SparsePageDMatrix::GetColDensity(unsigned long)+0xb0) [0x7f9ed837b60e]
  [bt] (8) /home/fis/Workspace/xgb/lib/python3.6/site-packages/xgboost-0.90-py3.6.egg/xgboost/./lib/libxgboost.so(+0x55a835) [0x7f9ed8471835]

Aborted (core dumped)

[02:08:13] WARNING: /home/fis/Workspace/xgb/xgboost/src/gbm/gbtree.cc:95: DANGER AHEAD: You have manually specified `updater` parameter. The `tree_method` parameter will be ignored. Incorrect sequence of updaters will produce undefined behavior. For common uses, we recommend using `tree_method` parameter instead.
[02:08:13] SparsePage::Writer Finished writing to tmptmp_.sorted.col.page
[02:08:13] SparsePageSource: Finished writing to tmptmp_
[02:08:13] SparsePage::Writer Finished writing to tmptmp_.col.page
[02:08:13] SparsePageSource: Finished writing to tmptmp_
[02:08:13] SparsePage::Writer Finished writing to tmptmp_.col.page
[02:08:13] SparsePage::Writer Finished writing to tmptmp_.col.page
[02:08:13] SparsePageSource: Finished writing to tmptmp_[02:08:13] SparsePageSource: Finished writing to tmptmp_

[02:08:13] SparsePage::Writer Finished writing to tmptmp_.col.page
[02:08:13] SparsePageSource: Finished writing to tmptmp_
Segmentation fault (core dumped)

I traced the commit back to 3f7e5d9c478e0cb94651e04a00a05fb9e2135c18 , the error is still here so it's not a recent regression.

I have another branch that didn't touch sparse page, somehow got an file open error in unittest:
[ RUN      ] HistIndexCreationWithExternalMemory.Test
[03:26:33] SparsePage::Writer Finished writing to /tmp/tmpdir.KwmJHQ/big.libsvm.cache.row.page
[03:26:33] SparsePageSource::CreateRowPage Finished writing to /tmp/tmpdir.KwmJHQ/big.libsvm.cache
terminate called after throwing an instance of 'dmlc::Error'
  what():  [03:26:33] /home/fis/Workspace/xgb/xgboost/dmlc-core/src/io/local_filesys.cc:209: Check failed: allow_null:  LocalFileSystem::Open "/tmp/tmpdir.KwmJHQ/big.libsvm.cache.col.page": No such file or directory
Stack trace:
  [bt] (0) ./testxgboost(dmlc::LogMessageFatal::~LogMessageFatal()+0x4a) [0x559135f87d80]
  [bt] (1) ./testxgboost(dmlc::io::LocalFileSystem::Open(dmlc::io::URI const&, char const*, bool)+0x2f1) [0x5591364a0dfd]
  [bt] (2) ./testxgboost(dmlc::Stream::Create(char const*, char const*, bool)+0x75) [0x55913648de33]
  [bt] (3) ./testxgboost(+0x83da99) [0x55913619fa99]
  [bt] (4) ./testxgboost(+0x83e969) [0x5591361a0969]
  [bt] (5) ./testxgboost(+0x83e663) [0x5591361a0663]
  [bt] (6) ./testxgboost(+0x83ebe6) [0x5591361a0be6]
  [bt] (7) ./testxgboost(+0x83eba2) [0x5591361a0ba2]
  [bt] (8) ./testxgboost(+0x83eb72) [0x5591361a0b72]

@trivialfis What kind of machine are you using to run tests? Is it a desktop or a cloud instance?
Here's what I think you should do:

Log all calls to C API functions of XGBoost that correspond to a single Python test.
Now write a C program that re-creates the exact sequence of the C API function calls. This is to remove Python so that it doesn't interfere with Thread Sanitizer.
Compile the C program with Thread Sanitizer enabled.
Run

@hcho3 Just a desktop with Ubuntu 19.04.  Let me figure something out tomorrow.

@hcho3 If you have any new progress on fixing dmlc-core threading bugs, feel free to ping me for testing.  Or if you have some concrete plans and want to divide and conquer, please let me know too. ;-)

@trivialfis I made a first round of fix that applies to ThreadedIter. I plan to handle the rest some time this week.

Just to update the error message
terminate called after throwing an instance of 'dmlc::Error'
  what():  [07:45:28] /home/fis/Workspace/XGBoost/xgboost/dmlc-core/include/dmlc/threadediter.h:218: Check failed: !producer_sig_processed_.load(std::memory_order_acquire):
Stack trace:
  [bt] (0) /home/fis/Workspace/XGBoost/xgboost/python-package/xgboost/../../lib/libxgboost.so(dmlc::LogMessageFatal::~LogMessageFatal()+0x4a) [0x7f729466ee82]
  [bt] (1) /home/fis/Workspace/XGBoost/xgboost/python-package/xgboost/../../lib/libxgboost.so(dmlc::ThreadedIter<xgboost::SparsePage>::BeforeFirst()+0x15e) [0x7f72946d0f06]
  [bt] (2) /home/fis/Workspace/XGBoost/xgboost/python-package/xgboost/../../lib/libxgboost.so(xgboost::data::SparsePageSource<xgboost::SparsePage>::BeforeFirst()+0xa9) [0x7f72946d12dd]
  [bt] (3) /home/fis/Workspace/XGBoost/xgboost/python-package/xgboost/../../lib/libxgboost.so(xgboost::data::SparsePageDMatrix::GetRowBatches()+0x74) [0x7f72946d9a44]
  [bt] (4) /home/fis/Workspace/XGBoost/xgboost/python-package/xgboost/../../lib/libxgboost.so(xgboost::BatchSet<xgboost::SparsePage> xgboost::DMatrix::GetBatches<xgboost::SparsePage>()+0x3d) [0x7f729468a3a9]
  [bt] (5) /home/fis/Workspace/XGBoost/xgboost/python-package/xgboost/../../lib/libxgboost.so(xgboost::data::SparsePageSource<xgboost::SparsePage>::CreatePageFromDMatrix(xgboost::DMatrix*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, unsigned long)+0x34d) [0x7f72946dcfc5]
  [bt] (6) /home/fis/Workspace/XGBoost/xgboost/python-package/xgboost/../../lib/libxgboost.so(xgboost::data::SparsePageSource<xgboost::SparsePage>::CreateColumnPage(xgboost::DMatrix*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool)+0x7f) [0x7f72946dac75]
  [bt] (7) /home/fis/Workspace/XGBoost/xgboost/python-package/xgboost/../../lib/libxgboost.so(xgboost::data::SparsePageDMatrix::GetColumnBatches()+0x5b) [0x7f72946d9b59]
  [bt] (8) /home/fis/Workspace/XGBoost/xgboost/python-package/xgboost/../../lib/libxgboost.so(xgboost::BatchSet<xgboost::CSCPage> xgboost::DMatrix::GetBatches<xgboost::CSCPage>()+0x3d) [0x7f729468a475]


Create or Update Footer

You can create or update the footer according to you.
In the footer, add some sections like one section can have some link to add some pages like the privacy policy, add a contact detail section, social media section, etc.

I want to work on this issue under JWOC, can you assign me this @codervivek5


add a command to report message

Description
Per discussion with admins, they would like a command to easily report a message. When clicked, the message URL will be sent to the moderator channel.
Type of change
Please delete options that are not relevant.

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update
[ ] Refactor or other

Checklist:

[x] I have performed a self-review of my own code
[x] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation

kk, will review


Use CircleCI 2.0

1.0 is being deprecated in the next month

@shentao your project is no longer running on CircleCI

Thanks! Finally got the time to go over the PRs and work on the release.


working example of glue crawler

Trying to get troposphere to give me a valid glue CFN, need s3 targets construction help
here is what I have
MyCrawler2 = t.add_resource(Crawler(
    "MyCrawler2",
    Name="testcrawler1",
    Role='mysecretrole'',
    DatabaseName=Base64('my_database'),
    Targets(
		'MyTargets',
		S3Targets = [S3Target(Path = 's3://bucket/something')]
	)
))

getting this error
TypeError: <class 'troposphere.glue.Crawler'>: MyCrawler2.Targets is <class 'troposphere.glue.S3Target'>, expected <class 'troposphere.glue.Targets'>

from troposhpere.glue import Targets, S3Target

MyCrawler = t.add_resource(Crawler(
               "GlueCrawlerName",
                Name=f"{crawler_name}",
                Role=GetAtt("GlueRole", "Arn"),
                DatabaseName=Ref(GlueDatabase),
                Targets=Targets(
                   S3Targets =  [S3Target(Path = f"s3://{target_bucket_name}")]
                )))


Add some advice for expensive background tabs

notions with results. Google Inc<EMAIL_ADDRESS>On Jun 22, 2017 2:30 AM, "Simon Pieters"<EMAIL_ADDRESS>wrote:

You can view, comment on, or merge this pull request online at:
https://github.com/operasoftware/devopera/pull/541
Commit Summary

Add some advice for expensive background tabs

File Changes

M src/blog/_posts/2017-06-22-opera-46.md
https://github.com/operasoftware/devopera/pull/541/files#diff-0 (7)

Patch Links:

https://github.com/operasoftware/devopera/pull/541.patch
https://github.com/operasoftware/devopera/pull/541.diff

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/operasoftware/devopera/pull/541, or mute the thread
https://github.com/notifications/unsubscribe-auth/ARQrqs0JPrPGbppt7cCHmm7Y1mm8o9y_ks5sGjQxgaJpZM4OCCSS
.


Align view causes core dump

Using Align View and selecting the -Y or +Y axis causes a core dump
To reproduce:

Load CAM/AdamPhillips/F2000C5_new120_025/FV_768x1152.bilinear.F2000C5_new120_025.cam.h0.2004-01.nc
Select Align View / -Y

VAPOR will abort

@clyne - I cannot reproduce this on 88a942090fafb2a3af3f3daf144bf10b723aa067 after importing the file, or loading its corresponding VDC on OSX.  Thoughts?

Who knows. Let's close it, but verify with actual installers.


"XML Parsing Error" in v1.28

After upgrading to v1.28, all of my sites started throwing the following error in the console:

This is also happening on https://umami.is.

Looks like this only happens in Firefox. It something to do with headers according to this, https://stackoverflow.com/questions/51000009/i-keep-getting-this-error-xml-parsing-error-syntax-error-but-still-the-website
The code still runs fine so no need to worry. I'll still fix it in the next release.

Ah, thanks!


Adding options for mutate tables and columns

Make it possible to create queries for mutating the table. In my case
used for migrations.
Options for tables: Create / Alter / Rename / Drop tables
Options for columns: Create / Alter / Rename / Drop columns

@eL-Prova I would really like to have this in my code, also for migrations :) but haven't test it yet. Can you add some documentation? Thanks.

@TCB13 yes I have multiple examples. I will try to create some documentation but it should first be merged back.

@usmanhalalit what's your opinion this features and the $this->builder... syntax?

Thanks @eL-Prova, but this seems like a massive change to me which I'm unable to manage at this moment. That's my problem though.

The latest tests were a success, it is a very good start.


add time span.

add time span data type support.

What is wrong with DateTime type? As a workaround, you could store a DateTime value compared to some reference value and calculate TimeSpan when you read it. Or use the Ticks property and store a number. You could also make a class where a DateTime property sets/gets the TimeSpan field and a separate TimeSpan property for what you need to do. In addition, use the Field attribute as follows:
...
class Foo
{
    [Ignore]
    public TimeSpan TS_Value { get; set; }

    [Field("WhateverFieldNameYouWant")]
    public DateTime DT_Value
    {
        get { return new DateTime (this.TS_Value.Ticks); }
        set { this.TS_Value = new TimeSpan(value.Ticks); }
    }
}

Or something like that...

@zhangsichu I like this patch but think the TimeSpan should be stored as a float and not a string to make queries possible.

Actually it should be stored as Ticks

@praeclarum I was going to work on this pull request, but it looks like support for TimeSpan's has already been implemented. Can you please confirm?
I'm looking at lines 2793, 2871, etc. of SQLite.cs.


HERE’S!!* WAY TO WATCH Jaguars vs Titans 𝗟𝗜𝗩𝗘-𝐒𝐭𝐫𝐞𝐚𝐦𝐬 NFL BrOaDcAsT Coverage 08 dec 2024

Jaguars vs. Titans odds, picks, spread, how to watch, live stream: Model reveals 2024 Week 14 NFL predictions
🔴➤👉✅NFL Watch Live🟢
🔴➤👉✅NFL Live Online🟢
🔴➤👉✅NFL Click Here Live🟢
SportsLine simulated Tennessee Titans vs. Jacksonville Jaguars in Week 14 of the 2024 NFL season 10,000 times
Two teams in the AFC South cellar square off in NFL Week 14 as the Tennessee Titans (3-9) host the Jacksonville Jaguars (2-10) on CBS and Paramount+. Will Levis and the Titans have shown flashes of competence but continue to lack consistency, following a Week 12 upset against the Houston Texans with a 42-19 loss to the Washington Commanders. Meanwhile, the Jaguars have already been eliminated from playoff contention and will be without starting quarterback Trevor Lawrence (concussion) in Week 14. You may be able to stream Sunday's game live on Paramount+, which you can now try free for the first seven days when you sign up here.
Kickoff from Nissan Stadium in Nashville is 1 p.m. ET. The Titans are 3-point favorites in the latest Titans vs. Jaguars odds via SportsLine consensus, and the over/under for total points scored is 39.5. Tennessee is a -167 money line favorite (risk $167 to win $100), while Jacksonville is a +141 underdog.
Paramount+ is the only place to watch every minute of every NFL on CBS game this season. A subscription also gives you access to other sports content including the UEFA Champions League and Europa League, NWSL, Serie A, and countless movies and shows. You can now get a free 7-day trial, so sign up right here.
🔴➤👉✅NFL Watch Live🟢
🔴➤👉✅NFL Live Online🟢
🔴➤👉✅NFL Click Here Live🟢
How to watch Jaguars vs. Titans
Titans vs. Jaguars date: Sunday, Dec. 8
Titans vs. Jaguars time: 1 p.m. ET
Titans vs. Jaguars TV channel: CBS
Titans vs. Jaguars streaming: Paramount+
Week 14 NFL picks for Jaguars vs. Titans
Before tuning into Sunday's Titans vs. Jaguars game, you need to see the NFL picks from SportsLine's advanced computer model. The model, which simulates every NFL game 10,000 times, is up well over $7,000 for $100 players on top-rated NFL picks since its inception. The model enters Week 14 of the 2024 NFL season on an incredible 24-10 betting hot streak on top-rated NFL picks this year. Longer term, it is on a 204-138 roll on top-rated NFL picks that dates back to the 2017 season and a 58-31 roll on top-rated NFL picks since Week 7 of 2022.
The model also ranked in the top 10 on NFLPickWatch four of the past six years on straight-up NFL picks and beat more than 94% of CBS Sports Football Pick'em players four times during that span. Anyone following at sportsbooks and on betting apps has seen strong returns.
For Titans vs. Jaguars, the model is backing Jacksonville to cover the spread. The Jaguars haven't won a true road game since Week 12 last season and are about to go up against the second-ranked defense in the NFL. However, the model is leaning on them to keep the score close against an equally inconsistent Titans team.
Quarterback Mac Jones will get the start for the Jaguars, and he is coming off of a two-touchdown pass effort with zero interceptions against the Texans in a very close 23-20 loss. Levis had the same TD-INT ratio against Washington but only completed 48.6% of his passes, although he will go up against a Jaguars passing defense that ranks dead last. In addition, Tennessee is 0-5 against the spread in its last five games at home. You may be able to stream the game here.
How to watch, live stream the NFL on CBS
Now that you know what to pick, get ready to watch your live local NFL on CBS games on Paramount+ on any device you want.


Record-based nodes support

Hello! While trying to render a graph like this one:
digraph structs {
    node [shape=record];
    struct1 [label="<f0> left|<f1> mid\ dle|<f2> right"];
    struct2 [label="<f0> one|<f1> two"];
    struct3 [label="hello\nworld |{ b |{c|<here> d|e}| f}| g | h"];
    struct1:f1 -> struct2:f0;
    struct1:f2 -> struct3:here;
}

I saw that currently record-based nodes are not supported by diagraph. I think it would be great to add this feature!

Hello, I will try to do something for the next version


Filter /any and /all

As a user, I can specify whether I want to filter by all fields passed or any field passed.

Issue moved to Project user stories board here.


Add colshape isPointIn

Adds isPointIn(x: number, y: number, z: number) to colshapes.
Related type def PR: https://github.com/altmp/altv-types/pull/1
Test JS:
import alt from 'alt-server';

const cs = new alt.ColshapeSphere(100, 100, 100, 5);

console.log(typeof cs.isPointIn);

[
	new alt.Vector3(100, 100, 100),
	new alt.Vector3(100, 106, 100),
	new alt.Vector3(97, 102, 98)
].forEach(point => {
	console.log(`isPointIn: ${point}? ${cs.isPointIn(point.x, point.y, point.z)}`);
});

Test Result:
function
isPointIn: Vector3{ x: 100, y: 100, z: 100 }? true
isPointIn: Vector3{ x: 100, y: 106, z: 100 }? false
isPointIn: Vector3{ x: 97, y: 102, z: 98 }? true

Why would you use 3 numbers instead of vector3?

I used 3 numbers to keep the consistency with the rest of the function calls dealing with coordinates:
vehicle constructor - https://github.com/altmp/altv-js-module/blob/dev/src/bindings/Vehicle.cpp#L43
spherical colshape constructor - https://github.com/altmp/altv-js-module/blob/dev/src/bindings/ColShape.cpp#L82
blip constructor - https://github.com/altmp/altv-js-module/blob/dev/src/bindings/Blip.cpp#L8
player spawn - https://github.com/altmp/altv-js-module/blob/dev/src/bindings/Player.cpp#L389
I do agree that a single Vec3 argument would make most sense though, I can change it if you would like.
Thanks.

@vadzz-dev let me know if you'd like me to change this to a single argument as opposed to the 3 numbers.
I would also be willing to submit PRs to favor vectors over 3 numbers in existing APIs if you're alright with the breaking changes


Defer completely to vcpkg to determine whether a cache is outdated

Currently, run-vcpkg tries to detect when the cache needs to be recompiled. It does so by doing a couple of approximations: OS runner version, vcpkg.json hash and vcpkg/ version hash. This isn't perfect of course, and causes a lot of redundant cache misses that require building every dependency again. There's also the problem using overlay ports that aren't part of the hash and don't trigger rebuilds (when they're modified) at all.
It seems like we could defer to vcpkg completely instead (this assumes manifest mode):

Restore binary cache, always (using a cache key without any restore keys of a simplified runner version: win22, ubuntu22, macos12, etc.).
Ask vcpkg if any ports need to be rebuilt with the triplet specified as input to the action => if yes, at the end of the job we will upload the cache
At the end ask vcpkg for a list of installed ports and check if it differs from the one in cache => if yes, a port was removed and we need to reupload the cache.
When uploading the cache we ask vcpkg which ports were installed, and we only cache those (so that ports that have been removed aren't uploaded)

This basically guarantees a working build (through vcpkg's ABI detection) with as few rebuilds as possible I believe.
Thoughts? I would be willing to help out as much as I can (given my limited TS experience probably not a lot but :shrug:). Thanks!

@quyykk the algorithm you propose is sensible, and it needs support from the vcpkg tool itself, for example:

how to ask vcpkg if anything needs to be rebuild.
how to ask vcpkg for the list of the installed ports and computing the difference with the one in the cache.

I think that having vcpkg doing all the cache management work would be the perfect solution that would be also working and beneficial for any scenario (e.g., on the developer machine and on GH runners and so forth).
Probably as @rcoup suggested, one improvement we can make is having a way to answer the question "does the current binary cache directory need to be saved?", and that could be done by asking, somehow, to vcpkg whether any package was not restored from the cache (the cache that has been restored by run-vcpkg before running vcpkg). In that case, run-vcpkg could be trying to replace the existing cache entry with the same key.

@lukka the vcpkg team seem fairly responsive to GitHub issues, maybe open a discussion there and ask for a recommended approach or decide on the best changes to vcpkg to support this use-case?

I asked around and it seems like everything already exists. The input needed is the requested feature and requested triplet.
If using the CMake toolchain (instead of calling vcpkg install manually), then CMAKE_BINARY_DIR is also required, but that can be read from the presets file.

how to ask vcpkg if anything needs to be rebuild.

$ vcpkg install --dry-run --x-feature="" --triplet="" --x-manifest-root="." --x-install-root="CMAKE_BINARY_DIR/vcpkg_installed"

This is using the CMake toolchain. If someone is not using the CMake toolchain, but running vcpkg install manually, then the last two parameters are not needed. If the root (where vcpkg.json is) is somewhere else then the path needs to be adjusted for the manifest root argument.
It returns a list of ports that need to be built and removed.

So I think it can work like this:
Manifest mode
(either from CMake or using vcpkg install)

Run the command above to get the list of every port used
Restore everything from cache
Run the command again to see if there have been changes
[...] now the build is done using run-cmake
If there has been a change (see 3), then a new cache needs to be saved with the ports from 1)

Manual mode (vcpkg install)
run-vcpkg needs to know about the ports that need to be installed, so they need to get passed to the action (no more executing vcpkg install ... in a separate step).

Restore everything from cache
Run vcpkg install --dry-run --triplet="" [package list] to get the list of changes (if any)
Install the packages with the same command but without --dry-run.
[...]
At the end if there was a change (see 2), then a new cache needs to be saved with the list of ports specified in the action input.

In both scenarios run-vcpkg could even save each port in a separate cache entry, to avoid unnecessary caches.

Ah shoot, this might not work depending on what exactly run-vcpkg caches. Is only the local binary cache restored? (~/.cache/vcpkg/archives/ on Linux)

@quyykk this is a detailed and actionable plan, thanks! run-vcpkg stores all the VCPKG_ROOT content along with the binary cache directory, so it is doable.
I wonder how the various solution compares:

current approach (cache key based on hash of vcpkg.json and vcpkg Git commit id): simple and effective.
this solution describe in detail above: quite complex, and more perfect than the previous one.
the solution suggested by the vcpkg team (last time I checked with them this is the suggested way): https://devblogs.microsoft.com/cppblog/vcpkg-accelerate-your-team-development-environment-with-binary-caching-and-manifests/

I think trying the last approach would be worth because it would be the simplest implementation and rely on vcpkg itself doing the cache job for us.

the solution suggested by the vcpkg team (last time I checked with them this is the suggested way):

So for vcpkg itself to store/restore from the Actions cache itself? from what I can tell, but I'm not sure. That does indeed sound much better.

@quyykk sorry for the late answer! Yes, I mean that, and thanks a lot for  creating the PR onto the vcpkg-tool repository. I look forward to consume the new cache mechanism as soon as it get merged in.

@quyykk thanks a ton for adding GHA support to vcpkg! This issue should be solved with this release:
https://github.com/lukka/run-vcpkg/releases/tag/v11.0
Let me know and thanks again!


Sparse accessor: typo fix

Sorry I made a mistake with accessor.sparse.values.byteOffset.
The value is not used at all in some applications, so the test didn't detect it.

Good catch! Thanks! Merged!


Added missing parameter in minify method

Otherwise this method will be unusable.

Correct is $target = NULL


Initial mesh support

Hi,
Thanks for making niivue-react! I’ve been using it for a while now, and it’s been awesome. Your diffing method is genius—way better than the way I was reacting (pun intented) to props changes in my old implementation. I ripped it out as soon as I see yours.
Recently, I needed mesh support for my project, so I put together this PR.
Right now, the support is pretty basic but gets the job done:

Load/remove mesh
Change mesh property (rgba255, visible, ...)
Change niivue's instance meshXRay

I’ll add more features as I need them for my project.
Cheers

Amazing! I’ll review later today or tomorrow.


Create Kuai Hinojosa

Hi @esquiva I just merged these changes

http://owaspsummit.org/Participants/Kuai-Hinojosa.html

If you can be more involved with the Summit organisation and Working Sessions please see the treads on the #owasp-summit-2017 slack channel


General issues with the generation

Some issues I've noticed are that the oceans are extremely small (like lakes in vanilla) and there are zero islands. There are also zero mushroom islands. Not sure if the trees being weird is a feature of the generation or if it's another plugin, but it doesn't work at all with Treefeller plugins.

Mushroom islands do generate, they're just very rare. There are 2 sources of mushroom islands, the actual islands that spawn in deep oceans, and large mushroom caves.
As for oceans being small, the default ocean size and frequency was done because people were saying the oceans were too big. If you'd like them bigger, you can just edit the config (decrease ocean threshold for larger oceans, raise ocean frequency for more common oceans)
Treefeller plugins are generally optimized for vanilla trees and will not work with the custom trees in TerraformGen (especially the larger ones), that part is correct. There's no forseeable solution for that one, short of a custom treefeller plugin which I don't have the time to make yet


added migration to remove double status changes for publication flows

Selected the offending status changes using the following query:
PREFIX pub: <http://mu.semte.ch/vocabularies/ext/publicatie/>
PREFIX prov: <http://www.w3.org/ns/prov#>

SELECT DISTINCT ?publicationFlow ?doubleStatusChange ?startedAt WHERE {
  GRAPH <http://mu.semte.ch/graphs/organizations/kanselarij> {
   {
      select ?publicationFlow COUNT(DISTINCT ?statusChange) as ?statusChanges where {
        ?statusChange a pub:PublicatieStatusWijziging .
        ?publicationFlow a pub:Publicatieaangelegenheid .
        ?publicationFlow prov:hadActivity ?statusChange .
      }
    }
    FILTER (?statusChanges > 1)
    ?publicationFlow prov:hadActivity ?doubleStatusChange .
    ?doubleStatusChange prov:startedAtTime ?startedAt .
    FILTER NOT EXISTS {
      ?publicationFlow prov:hadActivity/prov:startedAtTime ?startedAt2 .
      FILTER (?startedAt2 > ?startedAt)
    }
  }
} ORDER BY ?publicationFlow ?startedAt

Then created the DELETE statements using these results, see config/migrations/20220518141500-remove-double-publication-status-changes.sparql.
Tested locally with kaleidos-test db and seemed to work, but would like someone else to verify this as well.

Cause of the problem has been solved in frontend: https://github.com/kanselarij-vlaanderen/frontend-kaleidos/pull/1349. Going to merge this one.


added NYPL menus API

API to the NYPL historical menu transcription project data.
API link: http://nypl.github.io/menus-api/
Main project link: http://menus.nypl.org/

Many thanks :)


Effect macros not exported

With Kira 0.8 I had a custom Effect implemented in my library, mirroring how Kira effects were implemented. However, trying out 0.9/main it looks like this is not possible anymore due to the following macros not being exported:
handle_param_setters
read_commands_into_parameters
As well as frame being private.
I also checked the changelog if there is any information on how to migrate, but let me know if I missed anything! It does look like there will be less boilerplate with these changes.

Also got the following error:
error[E0433]: failed to resolve: use of undeclared crate or module `paste`
  --> crates\phonon-kira\src\direct_effect\mod.rs:15:1
   |
15 | / command_writers_and_readers!(
16 | |   set_parameters: DirectEffectParameters,
17 | |     set_panning: PanningEffectParameters,
18 | | );
   | |_^ use of undeclared crate or module `paste`
   |
   = note: this error originates in the macro `command_writers_and_readers` (in Nightly builds, run with -Z macro-backtrace for more info)

Which I was able to work around by simply adding paste to the dependencies of my library. But I don't think that's really the solution.

handle_param_setters and read_commands_into_parameters are not necessary for implementing Kira's traits; they're just there to reduce some boilerplate in Kira's code.  They're deeply tied to the naming conventions in Kira, so I'm not confident in their design as general-purpose macros, hence why they're not part of the public API. You can use Rust Analyzer to expand those macros to see what they're doing - everything you need is publicly available.
Frame isn't private, it's just been moved to the crate root.

In that case would it be worth it to create an example on how to implement a custom Effect? Because these would be extra barriers for new users.
Frame isn't private but the module frame is not, so I don't think I can use it.

Ah nevermind I missed the export of Frame below it


[BUG] DataGrid component shoots up CPU consumption and page slows down after 30 rows

Environment
Please provide as many details as you can:

Hosting type

[] Form.io
[x] Local deployment

Version: 5.2.0

Formio.js version: 4.13.9
Frontend framework: Angular
Browser: Chrome
Browser version: 105.0.5195.125

Steps to Reproduce

Create DataGrid Component with 10 columns (Mix of Text fields, date, Checkbox, numberfield)
Create 30 rows and above and fill data

Expected behavior

Datagrid component should work without issues or lag in page

Observed behavior

In text fields, spaces are deleted directly after typing them
In number fields, the comma often is ignored (it's used as the decimal delimiter)
When switching the fields with the tabulator, the cursor often just disappears and doesn't jump to the next field
In general, the form is lagging and slow

We're currently addressing a backlog of GitHub issues, and as part of this effort, some inactive issues may be marked as closed. This isn't a dismissal, but a step toward more efficient tracking.
If you feel the issue is still relevant, please re-open and we'll ensure it gets the attention it deserves. Your understanding is appreciated as we work to enhance our open-source responsiveness.


Contributing.md file added

Issue fixed, Contributing.md file added with.

@Harshit850 thanks for making a PR
The Contributing File should consist of a detailed step by step approach of how to contribute to the project so as any new comer will find it easy to contribute


[Dashboard]: Refactor the components to split the rendering and data awareness

Need to better split the rendering responsibilities of the Dashboard and data fetching components so that @stuckless can reuse them in other products with different backends.

Split out rendering Dashboard component from DataLoader component
CardRenderer splits the data state as well as the controls state and full card state
renderProp with the local card control state
How could the Maximo team reuse our value card from within their dashboards, could we allow a custom fetch function to be provided at the card level
Try getting the Value Card to work on Maximo Data from the Dashboard
Timeline December

@scottdickerson around this topic, I noticed that you moved the schema validation to the library level instead of the dashboard level. Would it be possible to allow some flexibility in the dataSource object to allow for customization? Currently in MWI dashboard we translate it differently to our backend. One example is that we support orderBy.  Also we might need other fields in the cards, for example we want to group some cards together, e.g: all KPI value cards will be called in a single API, etc..
What are your thoughts about this?

here's the external github link where the work will actually be done:
https://github.com/IBM/carbon-addons-iot-react/issues/653

@alronz let's chat and make sure whatever we do here satisfies your usecase as well as @sls-ca

@alronz just pushed a PR refactor, please take a look when you get a chance....

:tada: This issue has been resolved in version 2.18.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Add ProxyFromEnvironment in downloadFile method

Description
Add ProxyFromEnvironment in downloadFile method.
Issue reference
Please reference the issue this PR will close: #1043
Checklist
Please make sure you've completed the relevant tasks for this PR, out of the following list:

[x] Code compiles correctly
[ ] Created/updated tests
[ ] Extended the documentation

@Lichmaker Could you please check the lint failures.

@pravinpushkar  Glad to do it. Please check it. 😃   https://github.com/dapr/cli/pull/1049

@Lichmaker Can you please create a docs PR on how to use this ?
Potentially add it in this page? https://docs.dapr.io/operations/troubleshooting/common_issues/
cc @greenie-msft WDYT?

@Lichmaker Can you please create a docs PR on how to use this ? Potentially add it in this page? https://docs.dapr.io/operations/troubleshooting/common_issues/ cc @greenie-msft WDYT?

OK 😃  I'll try

@Lichmaker Thanks for the change to include docs PR !!


CLion C++ language mapping should not default to ObjectiveC

@tru wrote:

When installing and using this plugin in CLion it defaults to ObjectiveC scope (guessing it's because it's related to AppCode?) It should default to C++ in CLion or be configurable.

Any updates on this?

no update so far. i just have done some research and it is quite tricky to detect. when i find the time for the next release, it will be included.

looking for a solution as well


fix(build): ignore 403s when publishing and just continue

The most common publishing failure we currently see is a 403 where the target repo indicates that the artifact in question is already present in the repo. We don't know how that happens, but ignoring the error in such a case is perfectly fine as the artifact is actually already present.
This commit adds the --continue Gradle flag to the publish call to ignore the 403s. To avoid trying to upload something that's completely broken but would pass the build due to --continue, the build call has been put into a separate workflow step and therefore Gradle

:tada: This PR is included in version 4.3.2 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:

but ignoring the error in such a case is perfectly fine as the artifact is actually already present.

I think it is not perfectly fine because one task may upload multiple artifacts (e.g. model-server.jar and model-server.jar.md5). And if in a task the first artifact could not be published and the task aborted, then the second artifact would not be uploaded.

I think it is not perfectly fine because one task may upload multiple artifacts (e.g. model-server.jar and model-server.jar.md5). And if in a task the first artifact could not be published and the task aborted, then the second artifact would not be uploaded.

Maybe that formulation was a bit too strong. What we hope is that the likelihood for a complete release is higher. Right now, the releases with failures are usually severely broken as larger fraction of components is not uploaded at all. With --continue, we should get at least releases that are more complete and where chances are higher that clients can actually use them.


[Bug]: Unable to add multiple connections

What happened?
Unable to create and connect to switch between sources in Dataverse Dev Tools. It says Error occured and report in GIT Hub.
What version of the tool are you using?
Latest
What version of Visual Studio Code are you running?
Latest
Any relevant session id from VSCode?
5cf7d146-9ade-4ed2-afe4-b1bc790f42aa1674788605772
Relevant log output
Error occured in connectDataverse command.
Error code: undefined.
Error message: undefined.
Error stack: undefined.

Can you specify which type of authentication method you selected?

Hi Danish, I have used Built in authentication for Dataverse dev tools...However I faced this long back in Jan, not now, thanks for response....


handle lambdas

Currently java2ceylon ignores the parameter list of a Java lambda expression. Now that we have support for SAM conversions, all it needs to do is transform a skinny arrow into a fat arrow. This should be a pretty easy fix.

Added lambdas in 812fbc67729b2f. Is this enough?  I also added tests, because we need that feature in the upcoming 1.3.1 release.

@bjansen I fixed the single identifier conversion and the test in b22c7f707858ecca96.

Thanks folks!


Rendering error, some text flipped across page

In Firefox 36.0.1 / Ubuntu 14.04:

The header text "Advising Review Checklist" is flipped and on the bottom of the page. Same issue using Firefox 37.0.1 / Mac OS X Mavericks
Compare Adobe Reader 9.5.5 / Ubuntu 14.04:

Source file: https://jbeekman.nl/github-mozilla-pdf.js-issue-5946.pdf

Is there any information about this yes ?
I am having this issues also

I have yet to see a PDF.js bug I reported fixed.

Which version are you talking about ? I got the latest version: 1.2.109 and I still get the problem

Another:
https://issues.apache.org/jira/secure/attachment/12636888/D.1_signiert.pdf (signature missing)

A possible hint... there's a bug in the PDF specification: 12.5.5 Appearance Streams. Text is "Matrix shall be concatenated with A to form a matrix AA" should be "Matrix shall be concatenated to A to form a matrix AA" and the formula below must be changed too. This has been confirmed to be by somebody who has the PDF 2.0 specification.

I believe I found the cause and it is not related to my remark above. I enclosed the page content stream in q...Q, i.e. so that after rendering the page content, the graphics attributes would be reset to their original values, and now this file and the one from #4743 (with the code change mentioned by @Osukaru) are properly displayed.
So a code change would be needed that resets something before rendering annotations.

This code in core.js:
      return Promise.all([pageListPromise, annotationsPromise]).then(
          function(datas) {
        var pageOpList = datas[0];
        var annotations = datas[1];

        if (annotations.length === 0) {
          pageOpList.flush(true);
          return pageOpList;
        }

        var annotationsReadyPromise = Annotation.appendToOperatorList(
          annotations, pageOpList, partialEvaluator, task, intent);
        return annotationsReadyPromise.then(function () {
          pageOpList.flush(true);
          return pageOpList;
        });
      });
    },

if I've understood this correctly, you are creating a big list of operators with their parameters from the page content stream(s) and the content streams of the annotations. So if the page content stream(s) end with a flipped CTM (here: 1 0 0 -1 0 792 cm), the annotations will "inherit" that.

I'm a bit confused about this issue. We have https://github.com/mozilla/pdf.js/blob/master/src/display/canvas.js#L1839-L1846 which should take care of resetting the state already. I also tried adding explicit save and restore operators around beginAnnotations and endAnnotations at https://github.com/mozilla/pdf.js/blob/master/src/core/annotation.js#L396-L400, but also no luck. While @THausherr's explanation is very plausible, I'm unsure why these changes have no effect at all. The issue does appear to be not properly resetting the state before drawing annotations, but I don't know of anything more we have for canvas drawing to do that.
@yurydelendik @Snuffleupagus Do you perhaps have an idea for the above? It is affecting multiple PDFs in open issues, so it would really be nice to have this resolved.

What I meant is to save and restore before and after the main content stream. It is the main content stream that is the problem, i.e. if the main content stream doesn't reset all changes it has done.
I don't know enough about javascript, I remember you have some list of all operators with their parameters, ideally OPS.save should be inserted before everything, and OPS.restore at the end of the list, all that before you add the annotation content streams.

@jethrogb Are you the creator of the PDF file (https://jbeekman.nl/github-mozilla-pdf.js-issue-5946.pdf), and if so may we include it in the test-suite?

I'm not, and I don't know. I can try to get permission, but it'd be easier if you could recreate the test case.

Why was this issue closed?

I believe I found the cause and it is not related to my remark above. I modified the first file and enclosed the page content stream within q...Q, i.e. so that after rendering the page content, the graphics attributes would be reset to their original settings, and now this file and the one from #4743 (with the code change mentioned by @Osukaru) are properly displayed.
So a code change would be needed that resets something before rendering annotations.
The modified files: http://www.filedropper.com/pdfjs-5946unc http://www.filedropper.com/test-signatureunc

hey i have the same problem as stated above my electronic signature when viewed is seen upside down im in need of the modificatios you posted since the links are down if possible thank you


17 roles

Review was done see #94. I rewrote git history which ended up braking #94 as far as github is concerned so opened this pull request instead


ci(github): add codecov push

Codecov Report

:exclamation: No coverage uploaded for pull request base (main@f0c9f02). Click here to learn what that means.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main       #4   +/-   ##
=======================================
  Coverage        ?   99.51%
=======================================
  Files           ?        3
  Lines           ?      207
  Branches        ?       66
=======================================
  Hits            ?      206
  Misses          ?        1
  Partials        ?        0

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f0c9f02...1e349d4. Read the comment docs.


depth-nerfacto model invokes "PicklingError"

Hi!
Thank you for this amazing library first of all, I appreciate your effort to keep it up!
I would like to train a depth-supervised nerfacto model, but I am keeping getting "PicklingError" before training starts. Here is the full error code:
Traceback (most recent call last):   File "/home/virtualxdesign/.local/bin/ns-train", line 8, in <module>     sys.exit(entrypoint())   File "/mnt/d/NerfResearch/nerfstudio/nerfstudio/scripts/train.py", line 260, in entrypoint     main(   File "/mnt/d/NerfResearch/nerfstudio/nerfstudio/scripts/train.py", line 246, in main     launch(   File "/mnt/d/NerfResearch/nerfstudio/nerfstudio/scripts/train.py", line 200, in launch     process_context = mp.spawn(   File "/home/virtualxdesign/.local/lib/python3.8/site-packages/torch/multiprocessing/spawn.py", line 239, in spawn     return start_processes(fn, args, nprocs, join, daemon, start_method='spawn')   File "/home/virtualxdesign/.local/lib/python3.8/site-packages/torch/multiprocessing/spawn.py", line 188, in start_processes     process.start()   File "/usr/lib/python3.8/multiprocessing/process.py", line 121, in start     self._popen = self._Popen(self)   File "/usr/lib/python3.8/multiprocessing/context.py", line 284, in _Popen     return Popen(process_obj)   File "/usr/lib/python3.8/multiprocessing/popen_spawn_posix.py", line 32, in __init__     super().__init__(process_obj)   File "/usr/lib/python3.8/multiprocessing/popen_fork.py", line 19, in __init__     self._launch(process_obj)   File "/usr/lib/python3.8/multiprocessing/popen_spawn_posix.py", line 47, in _launch     reduction.dump(process_obj, fp)   File "/usr/lib/python3.8/multiprocessing/reduction.py", line 60, in dump     ForkingPickler(file, protocol).dump(obj) _pickle.PicklingError: Can't pickle <class 'nerfstudio.data.datamanagers.base_datamanager.VanillaDataManager'>: it's not the same object as nerfstudio.data.datamanagers.base_datamanager.VanillaDataManager
System information: WSL2 Ubuntu 20.04.6
Nerfstudio version: 0.3.1
Used dataset is in nerfstudio-data format
Full command:
ns-train depth-nerfacto --vis wandb --data {path_to_data} --output-dir {output_path} --experiment-name {experiment_name} --machine.num-gpus 2 --pipeline.datamanager.train-num-rays-per-batch 16384 nerfstudio-data --orientation-method vertical --center-method focus
I have no problem using nerfacto, or nerfacto-big , so I am wondering if I am missing something, or this is just a bug. I would be glad if you can help me about that. Thank you!

Hmm, thats odd. Can you try with only 1 gpu.

That worked! Works fine with 1 gpu. Thank you!


Improving Feedback section

IMPROVING #430

IMPROVING #430

Hi Harshita, changes are reflected the same on both prs as both are the same branches. So please don't create duplicate prs, you could've updated the original one as well. I've added changes to this branch so you can test your implementation. It's having multiple syntax errors. Also I'd suggest you to use material ui modal to make this better.


Build data for critic model

Hello, author! If I want to build my own critic model data, how should I build the preceding sententce?

I also want to know the answer and a complete description of the data creation process.

Hope you find this helpful. :)


Fix: Use a temp fixture test file for Zend/Paginator unit test

Fixes #283

changed title to be readable, fixed body to use close keyword, removed tasklist (tasklist should be used to mark tasks done/not done), rewrite commit message not to include references (those belong to pull request)