Prototype policy handler

Some rough prototype of what we were discussing in https://github.com/aspnet/Security/issues/1469
This would be a different direction than https://github.com/aspnet/Security/pull/1410
Thoughts @davidfowl @Tratcher
Example:
                auth.AddPolicyScheme("policy1", "policy1", p =>
                    {
                        p.DefaultScheme = "auth1";
                    })
                    .AddPolicyScheme("policy2", "policy2", p =>
                    {
                        p.AuthenticateScheme = "auth2";
                    })
                    .AddPolicyScheme("dynamic", "dynamic", p =>
                    {
                        p.DefaultSchemeSelector = c => c.Request.QueryString.Value.Substring(1);
                    })
                    .AddScheme<TestOptions, TestHandler>("auth1", o => { })
                    .AddScheme<TestOptions, TestHandler>("auth2", o => { })
                    .AddScheme<TestOptions, TestHandler>("auth3", o => { });

@HaoK,
Thanks for having already signed the Contribution License Agreement. Your agreement was validated by .NET Foundation. We will now review your pull request.
Thanks,
.NET Foundation Pull Request Bot

First glance: What's a policy scheme? You mean a scheme policy?

I'd rather put this project on hold until we can build out those samples we're discussing, I think they'll help us approach this more holistically.

@HaoK I'd wire up some more realistic policies (facebook, oidc, api, cookies). I'd like to see what they look like in sample code.
Besides the samples, I think this is on the right track and doesn't introduce any new concepts!

// Identity/default template
AddAuthentication(o => o.DefaultScheme = "Identity")
   // AddIdentityCookies() or the equivalent
   .AddPolicy("Identity", policy => {
       policy.DefaultScheme = "Identity.Application";
       policy.SignInScheme = "Identity.External"
   }))

// OIDC template:
AddAuthentication(o => o.DefaultScheme = "OIDCPolicy")
   .AddOIDC()
   .AddCookies()
   .AddPolicy("OIDCPolicy", policy => {
       policy.DefaultScheme = "Cookies";
       policy.ChallengeScheme = "OIDC"
   }))

// Social template (no identity)
AddAuthentication(o => o.DefaultScheme = "Social")
   .AddFacebook()
   .AddCookies()
   .AddPolicy("Social", policy => {
       policy.DefaultScheme = "Cookies";
       policy.ChallengeScheme = "Facebook"
   }))

// API doesn't really need a policy since its a single bearer already, but for completeness
AddAuthentication(o => o.DefaultScheme = "API")
   .AddJwtBearer()
   .AddPolicy("API", policy => {
       policy.DefaultScheme = "Bearer"
   }))

@HaoK I tweaked the samples slightly.
/cc @PinpointTownes @brockallen @leastprivilege

@HaoK we also need to see API and cookies in the same application. That's the scenario that requires switching based on some per request logic.

Request based scheme selection:
AddAuthentication("Dynamic")
   .AddJwtBearer()
   .AddCookies()
   .AddPolicy("Dynamic", schemes => {
       schemes.DefaultSchemeSelector = httpContext => IsRequestApi(httpContext) ? "Bearer" : "Cookies"
   }))

So is it safe to say this is a preferred option to where we were going in https://github.com/aspnet/Security/pull/1410 ?  If so I'll close that out as that seemed way more complex than this PR

@davidfowl thanks for pinging me.
I took a brief look and I'm not sure I like it (yet). Why doing that at the handler level? Why can't we integrate it with the APIs we already have? (e.g IAuthenticationSchemeProvider).
FWIW, I like the concept of authentication policies (because well, I'm the one who suggested that, initially :trollface:), but if you adopt it, then I'd expect seeing similar APIs to the ones we have for the authorization stack.

@PinpointTownes you mean you want it more of a first class new thing?  More like what we tried in https://github.com/aspnet/Security/pull/1410 then?

So I had a thought... why don't we just roll this concept into the base AuthenticationSchemeOptions/Handler.  We already sorta have this concept with RemoteAuthenticationOptions/SocialAuth where it forwards SignIn/Forbid to a Cookie.
Instead of adding a new thing, we could just bake this directly into AuthenticationSchemeOptions, and have the AuthenticationHandler base classes do the scheme forwarding automatically if set.
And then we wouldn't need a a new 'policy' for authentication, as scheme is already the equivalent of a named thing that is a Policy in Authorization.
So then you updating the examples a bit, using a new AddFowardScheme method that just adds the base handler implementation that can only forward
// Identity/default template
AddAuthentication("Identity")
   // AddIdentityCookies() or the equivalent
    .AddForwardScheme("Identity", options => {
       options.DefaultForwardScheme = "Identity.Application";
       options.ForwardSignInScheme = "Identity.External"
   }))

// OIDC template:
AddAuthentication("Cookies")
   .AddOIDC()
   .AddCookies(options => options.ForwardChallengeScheme = "OIDC")

// Social template (no identity)
AddAuthentication("Cookies")
   .AddFacebook()
   .AddCookies(options => options.ForwardChallengeScheme = "Facebook")

// API doesn't really need a policy since its a single bearer already, but for completeness
AddAuthentication("Bearer")
   .AddJwtBearer()

Request based scheme selection:
AddAuthentication("Dynamic")
   .AddJwtBearer()
   .AddCookies()
   .AddForwardScheme("Dynamic", schemes => {
       schemes.DefaultForwardSchemeSelector = httpContext => IsRequestApi(httpContext) ? "Bearer" : "Cookies"
   }))

@PinpointTownes you mean you want it more of a first class new thing? More like what we tried in #1410 then?

Yep. The other PR looks better, IMHO (and more consistent with existing concepts in the authorization stack).

Yeah but its also pretty complicated, personally I prefer the direction this PR is going, but its possible there are still some things we added in the other PR that are worth keeping

So summarizing my current thinking after playing with this for a while:
I don't think we need a new concept of AuthenticationPolicy.  An authentication scheme is exactly what we want already.  It has exactly the same operations a policy has: Authenticate/Challenge/SignIn[Out]/Forbid.  The problem today is its not easy at all to mix and match the operations from multiple authentication schemes for a single authorization.  But we can solve that by simply moving towards registering a "Virtual/Dynamic" authentication scheme, that is able to combine the operations of other schemes arbitrarily (and dynamically).
That's what kept bugging me about the other PR trying to introduce authentication scheme, it felt like a slightly different flavor of what we already had.
The benefit of this new pattern of using "virtual/dynamic" authentication schemes, is that we can move away from needing to support multiple authentication schemes entirely.  For any situation where executing multiple authentication handlers is necessary in a challenge/authenticate, you just plug in a custom authentication scheme to aggregate it explicitly.
Thoughts @davidfowl @Tratcher @PinpointTownes ?

Overriding classes is a much higher usage barrier. We have some very common scenarios today like JWT + Cookies that shouldn't require that level of customization.

We have some very common scenarios today like JWT + Cookies

That doesn't require a custom authentication handler, its only if you wanted to challenge more than one scheme, or authenticate more than one scheme and aggregate.  So like if you wanted to challenge Windows + Basic or multiple Bearer schemes at once for example.
The JWT +Cookie example would look like above, I'm not proposing doing nothing, just not adding anything resembling a 'policy':
AddAuthentication("Dynamic")
   .AddJwtBearer()
   .AddCookies()
   .AddVirtualScheme("Dynamic", schemes => {
       schemes.DefaultForwardSchemeSelector = httpContext => IsRequestApi(httpContext) ? "Bearer" : "Cookies"
   }))

That's an interesting example. So AddVirtualScheme is something we'd provide out of the box?  (or we call it AddDynamicScheme and you don't have to specify the name)

Yes I'm proposing we go with the idea that we make it easy to virtualize/forward + resolve schemes dynamically (the last set of examples I showed above).  But no, you NEED to give it a real authentication scheme name.  It just doesn't have to do anything, we'd either add a new base handler that did this forwarding logic, or we roll it into the existing base handler class and make it non abstract (effectively the same thing)

Basically I'm arguing schemes are already policies...not that we don't want to do all the stuff in this PR :)

I'm guessing we need to create a VirtualAuthenticationHandler since we don't implement all the interfaces by default anymore...

@HaoK I agree and I'm on the same page.

That doesn't require a custom authentication handler, its only if you wanted to challenge more than one scheme, or authenticate more than one scheme and aggregate. So like if you wanted to challenge Windows + Basic or multiple Bearer schemes at once for example.

This scenario was supported in 1.0 and it's now painful to do in 2.0 (the remarks on the announcement tickets are eloquent). I believe it should be supported natively and shouldn't require adding custom code.
The problem with the approach you suggest is that it's all or nothing (e.g all the challenge/forbid/signin/signout/authentication schemes points to "Cookie" or "Bearer"). Having a policy-based system would allow much more flexibility.
For me, something like that would be the ideal API:
services.AddAuthentication(options =>
{
    options.AddIdentityPolicy("Cookies");

    options.AddPolicy("JWT tokens only", policy =>
    {
        policy.UseAuthenticateShemes(JwtBearerDefaults.AuthenticationScheme);
        policy.UseChallengeSchemes(JwtBearerDefaults.AuthenticationScheme);
    });

    options.AddPolicy("JWT tokens + Data Protection tokens", policy =>
    {
        // Authenticate requests using both the JWT and the aspnet-contrib handlers.
        policy.UseAuthenticateShemes(
            JwtBearerDefaults.AuthenticationScheme,
            OAuthValidationDefaults.AuthenticationScheme);

        // Both the JWT and aspnet-contrib are "Bearer" handlers. Apply a single
        // challenge to ensure no duplicate "Bearer" WWW-Authenticate is returned.
        policy.UseChallengeSchemes(JwtBearerDefaults.AuthenticationScheme);
    });

    options.DefaultPolicySelector = context =>
    {
        if (context.Request.Path.StartsWithSegments("/legacy-api"))
            return "JWT tokens only";

        if (context.Request.Path.StartsWithSegments("/api"))
            return "JWT tokens + Data Protection tokens";

        return "Cookies";
    };
})

.AddJwtBearer(...)

.AddOAuthValidation();

You could even add the [Authenticate] attribute I suggested here to make the policy story super nice:
[Authenticate("Bearer")]
public class ApiController : Controller
{
    public IActionResult ActionWithOptionalAuthentication() { ... }

    [Authorize]
    public IActionResult ActionWithRequiredAuthentication() { ... }

    [Authorize(Policy = "HasSuperPrivilege")]
    public IActionResult ActionWithRequiredAuthenticationAndSpecifiedPolicy() { ... }
}

[Authenticate("Cookie")]
public class WebController : Controller
{
    public IActionResult ActionWithOptionalAuthentication() { ... }

    [Authorize]
    public IActionResult ActionWithRequiredAuthentication() { ... }

    [Authorize(Policy = "HasSuperPrivilege")]
    public IActionResult ActionWithRequiredAuthenticationAndSpecifiedPolicy() { ... }
}

@PinpointTownes I think we're splitting hairs TBH.

Having a policy-based system would allow much more flexibility.

How doesn't this design achieve the same thing? It looks identical, what am I missing?

How doesn't this design achieve the same thing? It looks identical, what am I missing?

My proposal allows assigning multiple schemes per request and different authenticate/signin/signout/challenge/forbid schemes per request. @HaoK's approach doesn't.

Yes my hope is to drop any explicit support for automatic routing of authentication/challenge to multiple schemes entirely, leave it as a pure extensibility (code + custom authentication handler scenario), so there's no expectation it just works.  If we get repeated asks to make it easier to do complex things that involve things like merging multiple schemes for authentication or complex challenge behavior, we can always add it later.

Yes my hope is to drop any explicit support for automatic routing of authentication/challenge to multiple schemes entirely, leave it as a pure extensibility (code + custom authentication handler scenario), so there's no expectation it just works.

#fail.

Challenging multiple authentication schemes today is basically unsupported in many scenarios anyways, it is almost impossible to get it to work without dealing with quirks.  This puts things in a much better state going forward.

Multiple authentication scheme support today is basically unsupported anyways, it is almost impossible to get it to work without dealing with quirks anyways.

It was perfectly supported in 1.0, where you could mix Basic + Bearer, Basic + Negociate, Basic + Negociate + API Keys, etc...

You can still accomplish a virtual scheme that represents multiple schemes, you just need to plug in an auth handler that does it...  It's not going to be a lot of code.

Nothing was 'perfectly supported' in 1.0.  Everything barely worked...and everything broke in minor regressions with any changes...  Its 'perfectly supported' in 2.0 as well, when you write an auth handler which represents your "policy"

We don't ship basic auth today do we?

Nothing was 'perfectly supported' in 1.0.

Ask @Tratcher, mixing NTLM + Bearer (or any other non-interactive scheme) used to work just fine (as designed).

You mean how you can have multiple automatic authenticate = true in 1.0?

That was totally not how it was designed...we just had no way to enforce single automatic authenticate easily

We don't ship basic auth today do we?

Sure you do. Both IIS and HttpSysServer support it.

Yes and the hosts are already effectively using schemes as described in this PR

Define can't mix?

If you want to mix "Windows" with "Bearer", you would have to add your own "WindowsBearer" that authenticated/challenged both yes.  This is no different than how things stand in 2.0 anyways.  So we can split that into its own issue/PR if that's the case.

That was totally not how it was designed...we just had no way to enforce single automatic authenticate easily

You don't get it, obviously. In 1.0, multiple non-interactive authentication middleware could deliberately apply their challenges to a same response. That's exactly the reason why JwtBearerHandler.HandleUnauthorizedAsync() returns false in 1.x (again, ask @Tratcher if you don't trust me...).

I get it, you like the old chaining behavior we explicitly killed in 2.0.  You can chain, but you do yourself in IAuthenticationHandler.Challenge

This is no different than how things stand in 2.0 anyways.

Stop talking about 2.0, we all know the situation is not ideal and that many scenarios are no longer supported. Mixing NTLM and Bearer was supported in 1.0 and is not in 2.0. Call that whatever you want, but it's a regression.

With a huge difference. You have to explicitly define a policy asking for multiple schemes to be used. In 1.0, most authentication middleware were configured to use automatic challenge.
options.AddPolicy("JWT tokens + Data Protection tokens", policy =>
{
    // Authenticate requests using both the JWT and the aspnet-contrib handlers.
    policy.UseAuthenticateShemes(
        JwtBearerDefaults.AuthenticationScheme,
        OAuthValidationDefaults.AuthenticationScheme);

    // Both the JWT and aspnet-contrib handlers are "Bearer" handlers. Apply a single
    // challenge to ensure no duplicate "Bearer" WWW-Authenticate is returned.
    policy.UseChallengeSchemes(JwtBearerDefaults.AuthenticationScheme);
});

Its really not that much more code, and merging principles really should be handled by the app itself:
services.AddAuthentication("JwtData")
            .AddJwtBearer()
            .AddOAuthValidation()
            .AddVirtualHandler<JwtDataAuth>("JwtData")

public class JwtDataAuth : VirtualAuthenticationHandler {
   public override async Task<string> HandleAuthenticate()
      => MergeYourPrincipals(await Context.AuthenticateAsync("Bearer"), await Context.AuthenticateAsync("OAuthValidation");

     public override Task HandleChallengeAsync() => Context.ChallengeAsync("Bearer");
   }
}

and merging principles really should be handled by the app itself

There's already code that handles that in ASP.NET Core, there's really no reason to do that yourself, is it?
I really hope you'll end up with a better design for such simple scenarios. Just saying folks "hey, you know what? Write custom code!" is not really satisfying.

That merging principal code was something we totally made up for this (we reorder things, and it also cannot be used with WindowsPrincipals at all)  So if you try to use it with Windows auth it will likely explode.

So if you try to use it with Windows auth it will likely explode.

The "merging code" works with Windows identities. It's just that the resulting principal is a simple ClaimsPrincipal so things like IsInRole don't work properly in this case.

Replacing with new iteration PR


⚠ Jenkins Plugins Artifactory has degraded performance

In 71d9db8, Jenkins Plugins Artifactory (https://get.jenkins.io/plugins/artifactory/) experienced degraded performance:

HTTP code: 200
Response time: 1181 ms

Resolved: Jenkins Plugins Artifactory performance has improved in 50344b0 after 25 minutes.


Add support for Xcode 9

Compiles but doesn't actually work yet (the Float <-> NSNumber conversions crash even when building as Swift 3.2).

Now successfully builds and passes tests with Xcode 8.3 and Xcode 9 in both swift 3 and swift 4 modes, so I guess it's ready for review.

Going to file a radar about the NSNumber as! Float.

It looks like some of the CI builds (tvOS and watchOS) are failing to compile with errors, while other builds are just causing the compiler to segfault.

Should we standardize on @objc rather than @objc dynamic?

@objc without dynamic doesn't work.

That's very good to know (and surprising)

My impression was that dynamic prevents the compiler from allowing the method to be invoked by anything other than dynamic dispatch, while @objc doesn't make that same guarantee if dynamic dispatch isn't required (e.g. if the method is not actually ever invoked from Objective-C code).

awesome pull request. Thanks for rolling it in.

Remaining CI failures appear unrelated (test failure in the sync progress notifier and failing to do anything at all in the device test).

Any update on this? Should we build from TG's branch for Xcode 9?

@cnstoll, I build branch by self using cocoapod and it work.

Does that mean with Swift 4 we're going to have to add both @objc along with dynamic to all our properties?
Yuck.

Yes, with Swift 4, exposing things to the obj-c runtime has been decoupled from disabling static dispatch and we need both parts.
Property behaviors will hopefully make it possible for us to switch to a pure-Swift solution and not need @objc or dynamic, but those aren't coming in Swift 4.


Updates post chatbot refactor

Preview: https://virtual-assistant-pr-virtual-assistant-116.surge.sh


[doc] Full GUI userguide rewrite

GUI tests are failing. Is this a known issue or should they run?

GUI tests are failing. Is this a known issue or should they run?

@bruntib I know about it, sometimes it fails for no reason. The virtual machine unable to start selenium server. I rebased my branch, so it will rerun the test. I hope now it will pass.

@bruntib I know about it, sometimes it fails for no reason. The virtual machine unable to start selenium server. I rebased my branch, so it will rerun the test. I hope now it will pass.

@bruntib Okay, now it is passed.

@bruntib Okay, now it is passed.


DbaSpn Features for v1.1

Is this a feature OR bug:
feature

Is it possible to check for delegation in the get? should it? We should in Test.
User reports that he works at place with hosted database servers and their ISP gives them "server101013df5" so they make CNAMEs like sqldev01 to make connecting more human friendly, while DNS management stays... manageable. They add SPNs to support this setup, and it works. CNAMEs generally fail otherwise. @niphlod does that line up with what you figured out?
A NoDelegation switch to skip delegation for whatever reason, then a section on Set and Remove webpages about this along with links to why it's the recommended from Microsoft.

@ctrlbold : if an SPN is set on the A record that matches the server running SQL Server, connecting to a CNAME of that A results in a kerberos connection, without setting the SPN to the CNAME itself.

all done


Fix: Snapshot size

By default limit PartInstances and PieceInstances in rundown snapshot to only non-reset, or reset but no older than 24 hours;
Adds extra parameter to the method and the REST endpoint to get the full snapshot with older instances

Codecov Report

Merging #44 (95f8ded) into develop (aec5ce0) will decrease coverage by 0.03%.
The diff coverage is 37.50%.

@@             Coverage Diff             @@
##           develop      #44      +/-   ##
===========================================
- Coverage    68.32%   68.28%   -0.04%
===========================================
  Files          217      217
  Lines        14714    14732      +18
  Branches      3468     3472       +4
===========================================
+ Hits         10053    10060       +7
- Misses        4483     4494      +11
  Partials       178      178

Impacted Files
Coverage Δ

meteor/lib/api/shapshot.ts
100.00% <ø> (ø)

meteor/lib/api/userActions.ts
100.00% <ø> (ø)

meteor/server/api/userActions.ts
38.42% <0.00%> (ø)

meteor/server/api/snapshot.ts
13.53% <9.52%> (-0.11%)
:arrow_down:

meteor/server/api/playout/lib.ts
80.99% <100.00%> (+0.15%)
:arrow_up:

meteor/server/api/rundown.ts
63.12% <100.00%> (+0.42%)
:arrow_up:

meteor/server/api/systemTime/systemTime.ts
62.88% <0.00%> (-1.04%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update aec5ce0...95f8ded. Read the comment docs.


C++ support

Hi,
I am just wondering what is your intension for C++ support?
We are implementing some C++ project that would like to interact with netopeer2.
Thanks

Since netopeer2 is just a binary, I have no idea what you mean by C++ support and how are you "interacting" with netopeer2. Maybe you mean sysrepo?

Sorry, I meant sysrepo. I saw the posts about the c++ bindings. Thanks


Previews don't work, peek RuntimeError

Yazi version: 0.3.3 (Arch Linux 2024-09-05)
Plugin commit version: db1488358941a2bc9918fa91c304d6724a0bb608
When I hover over a .zip, I get the following log.
  2024-12-02T00:39:01.627007Z ERROR yazi_plugin::isolate::peek: RuntimeError("[string \"ouch\"]:39: attempt to call a nil value (field 'Text')\nstack traceback:\n\t[string \"ouch\"]:39: in function <[string \"ouch\"]:3>")
    at yazi-plugin/src/isolate/peek.rs:50

Here are my yaml config contents. It's very barebones.
# keymap.toml
[[manager.prepend_keymap]]
on = ["C"]
run = "plugin ouch --args=zip"
desc = "Compress with ouch"

# yazi.toml
[manager]
sort_by = "modified"
sort_reverse = true

[plugin]
prepend_previewers = [
	# Markdown glow
	{ name = "*.md", run = "glow" },

	# Archive previewer
	{ mime = "application/*zip", run = "ouch" },
	{ mime = "application/x-tar", run = "ouch" },
	{ mime = "application/x-bzip2", run = "ouch" },
	{ mime = "application/x-7z-compressed", run = "ouch" },
	{ mime = "application/x-rar", run = "ouch" },
	{ mime = "application/x-xz", run = "ouch" },
]

prepend_preloaders = []

[opener]
extract = [
	{ run = 'ouch d -y "%*"', desc = "Extract here with ouch", for = "windows" },
	{ run = 'ouch d -y "$@"', desc = "Extract here with ouch", for = "unix" },
]

I seem to be experiencing the same problem, after upgrading today.
The zip files that can't be previewed are from other people, the ones I created myself can be previewed normally. Maybe it's an encoding issue?

I reverted to before the update and the problem disappeared.
I think it's supposed to be some library update that broke yazi's recognition of some encoded files

Looks like the same as #12

You're right. I completely missed the = and only saw the <.
Solved.


Regression in debug_full_compile_millis for iOS compile time benchmarks

See https://flutter-flutter-perf.skia.org/e/?begin=1627670324&end=1627940405&keys=X2fa997fe887b92a9e2bf8d4f4032bec4&num_commits=50&request_type=1&xbaroffset=25271
My best guess is this PR: https://github.com/flutter/flutter/pull/87362
Regression is in the ~10% range.
@jmagman

@zanderso The regression is expected, I don't see another way to solve https://github.com/flutter/flutter/issues/85713.

Closing per comment from @jmagman


Fixed issue template name field

Fixed issue template name field Framewor7 -> Framework7 for React, Svelte and Vue (The one in the Core version was fixed already six months ago).
I would like to point out that the same typo is also present in the name of these four files:

Thanks!


deployment error (size limit) on azure for nextjs app (single static web app )

The content server has rejected the request with: BadRequest
Reason: The size of the function content was too large. The limit for this Static Web App is 104857600 bytes.

Tried Soln#1: tried to add cache clearing commands to package.json & github/workflows/ azure-static-web-apps-salmon-ocean-0c2650810.yml file

this problem is resolved by taking below action.
Set "output" in next.config.js to "standalone"


Experimental instance: Features from different extensions are accessible across extensions

When I run debugging, the experimental instance is opened, but there are features from a different extension project that shouldn't be accessible.
See:

How can I get rid of the features that are not related to the extension being debugged?

Hi @i-m-luke, the extensions deployed while debugging are not removed at the end of the session. You can clear them out manually by removing them in %LOCALAPPDATA%\Microsoft\VisualStudio\17.0_Exp\VSExtensions. Alternatively resetting your Experimental instance from the shortcut in Start menu should also remove all extensions from it.

 What shortcut are you refering to? I tried restart the debug process (CTRL + SHIFT + F5) but it didn't help.

@i-m-luke - by "Start menu" we mean going to the Windows start menu and search for the command to reset your experimental instance. This is a command line tool that comes with your VS install:

Note that resetting will only clear your experimental instance from previously debugged extensions. It will not clear extensions that you have installed to your main instance of VS. Hope this helps.

@tinaschrepfer Thanks for the clarification.


Critical success or failure hightlighting isn't working

Changes to the ChatMessage class in foundry core broke the highlightCriticalSuccessFailure(message, html, data) method.

Fixed in 5223420 . Will be available in next release.


Add reply to guid to data sent to clients

Not merging yet until we can investigate how to not use columns with macos versions that don’t support a specific column

I've implemented this conditionally based on os version


chore: add script to generate caretaker presubmit scheduler script

@mmalerba @josephperrott @tinayuangao @andrewseguin
This is my first pass at writing a low-effort presubmit scheduler. I decided it to have it write a script so that you can make any local modifications necessary before running it (such as adding --cherrypick or omiting certain PRs). Let me know if you have any ideas to improve it.

@jelbourn please rebase


Uncaught TypeError: hljs.getLanguage(...).rawDefinition is not a function

Describe the bug
Receiving the following error with Highlight.js 9.16.2.
Uncaught TypeError: hljs.getLanguage(...).rawDefinition is not a function

Reporting this on behalf of @mattwhocn.  Was originally filed against Highlight.js:
https://github.com/highlightjs/highlight.js/issues/2279
To Reproduce
Steps to reproduce the behavior:

Try using react-syntax-highlighter with Highlight.js 10.16.2
See issue I imagine (see additional context below)

Sorry I don't have more specific steps.
Expected behavior
The library works as expected without throwing the error:
Uncaught TypeError: hljs.getLanguage(...).rawDefinition is not a function

Desktop:
Dunno. Pretty sure this is not environment specific. (see addl. context)
Smartphone:
Dunno.  Pretty sure this is not environment specific. (see addl. context)
Additional context
You have created a very custom loading process for languages.  Newer versions of Highlight.js have stronger interconnections between languages.  cpp needs to be registered BEFORE arduino can be registered as they now depend on each other at run-time (during registration).
Not sure what might need to change to fix this, but wanted to file it here as it seems to be a problem with how you all are packaging/loading the languages.

Though actually 9.15.10 has the same dependency:
function(hljs) {
  var CPP = hljs.getLanguage('cpp').exports;
	return {

So now I guess I'm a little confused.

Ok, I'm thinking this is a lowlight issue now, sheesh.  Off to open an issue there. :)  Will link it here just incase.

https://github.com/wooorm/lowlight/issues/19

Curious: How does build-languages-highlightjs.js guarantee the correct load order though?  I don't see any mention of Requires or re-ordering the languages to load in the correct order.
Arduino will below up if it's loaded before CPP.  So not sure how this is working now?

This is what I'd expect you to see:
https://github.com/conorhastings/react-syntax-highlighter/issues/10

Should work now after updating to lowlight 1.13.0

Any idea on my ordering question?

@yyyc514 there is ordering logic, is this defined anywhere if it is i can build it in

Well it may change in the future but right now the language files are annotated in the top comment. “Requires”
Dependencies should be loaded first. Though now I’m already thinking about how we might remove this requirement.

With refractor/prism it's more clear (as in, separated) than with hljs. There I use babel to inspect the languages to require and go from there.
I'd suggest using something like require or import for hljs? That's useful for tools like us and built tools internally

Long term we likely will. First we need new build system which I’ve been working on. Though eventually that gets into how do we do duplication in modules if we’re resolving dependencies at compile time not runtime.
https://github.com/highlightjs/highlight.js/issues/2248

Is just ripping ordering from index.js not trivial?

First we need new build system which I’ve been working on

That would be awesome! Btw, I think the questions you ask there are supported by build tools like webpack (async loading), Conor is doing that here!

We could indeed also introduce a build step with babel in lowlight, similar to refractor, that detects e.g., hljs.getLanguage calls and creates language bundles.
Similar to the syntax tree, I’d prefer these features to go in hljs and prism though, to benefit the wider ecosystem and reduce the work I need to do!

  Once we have a parse tree output you could really slim down a lot (by requiring us vs rewriting us), and then I don't think you'd run afoul of this type of issue in the future - since really your codebase would mostly disappear.
I'm not sure why react-syntax-highlighter needs such a complex build system for HLJS.

Again, that is exactly what I want.
react-syntax-highlighter has a build system because it solves what your linked issue is about

But our NPM package already has requirable modules... are they going to ALL that trouble just to get the registerLanguage included in the require?  No, because then they still have to go and register it manually... I'm not sure I get the point.
Not sure why they just couldn't require our NATIVE packaged languages directly rather than wrap them all.

Again, that is exactly what I want.

I've been working on other things but when I have a PR with the tree stuff I'll ping you and you can play with it and give feedback.

Closing this issue as we've refactored our lowlight vs refractor detection logic and are no longer calling getLanguage() anywhere.
Did you ever get a satisfactory conclusion to this issue elsewhere, @yyyc514 ?

I only skimmed but I think my answers remain the same as before:

Build and load a monolith (letting us handle the dependencies)
Use the Requires: meta-data in the languages files to build your own dependency tree.
Use the order of languages in the index file to decide "proper" load order.

Long-term we might switch to modules everywhere and move away from run-time dependencies but that's probably a 2021 thing, and I don't see a big rush. Newer work we're doing (like pulling ECMAscript out into a dependency of JS, TS, Livescript, Coffeescript, etc) is all module based rather than run-time to avoid these issues entirely.


Added context menu feature

I thought I should do something, and thought to add simple context menus to the CodeView widget.
(Honestly, what description you want here?)
One screenshot and code snippet is more than enough.

import tkinter as tk
import pygments
from chlorophyll import CodeView

root = tk.Tk()
editor = CodeView(root, undo=True,
        font = "Iosevka\ Curly\ Light\ Extended 14",
        lexer=pygments.lexers.PythonLexer, color_scheme="ayu-dark",
        context_menu=True
    )
editor.context_menu.add_separator()
editor.context_menu.add_command(label="A custom command", command=lambda:print("lame"))

editor.pack(expand=True, fill='both')
root.mainloop()

Also, @Moosems I and @rdbende were thinking about whether we should set undo=True (since it's like the bare minimum for Text widgets) or leave it to False for Tkinter's "90's compatibility". What are your opinions?

It's almost always turned on anyway so yeah, that seems good

@Moosems thanks for your input...
I'd be glad if you take a deep look at my PR and point out any mistakes

In my opinion, it would be better to have a read-only property, that abstracts the _context_menu field, and lazy-initializes the menu instance (i.e. only, when needed/the property is accessed). The context_menu option could be renamed to default_context_menu, so that you can also get an empty menu, but there's an option to fill it with the default items.

Honestly it is the default_context_menu so I don't know, you make the call

@rdbende, it seems that your commit with if super().tk.call("tk", "windowingsystem") == "aqua": doesn't work for me:
$ python3 example.py
Traceback (most recent call last):
  File "example.py", line 11, in <module>
    codeview_01.context_menu.add_separator()
  File "E:\Programming\_github\chlorophyll\chlorophyll\codeview.py", line 108, in context_menu
    if super().tk.call("tk", "windowingsystem") == "aqua":
AttributeError: 'super' object has no attribute 'tk'

I changed super().* to self.* and it works like a charm.

Moreover, I also realized that if you create a CodeView with default_context_menu=True, you ought to be expecting the context menu to be created, and not call CodeView.context_menu.xx() to initialize the object. Does anyone agree with me?
That being said, shouldn't we have if self._default_context_menu: self.context_menu to initialize the instance right after self._default_context_menu = default_context_menu?

That being said, shouldn't we have if self._default_context_menu: self.context_menu to initialize the instance right after self._default_context_menu = default_context_menu?

Oh, yeah that's missing

Now, check for any mistakes... 🤞

Unrelated, but I still don't like pyperclip
It now says, that it can't find a clipboard implementation for my system (probably because it was running inside a distrobox with nothing but base Debian and tkinter installed)
Exception in Tkinter callback
Traceback (most recent call last):
  File "/usr/lib/python3.11/tkinter/__init__.py", line 1948, in __call__
    return self.func(*args)
           ^^^^^^^^^^^^^^^^
  File "/run/host/tmp/chlorophyll/chlorophyll/codeview.py", line 163, in _copy
    copy(text)
  File "/run/host/tmp/chlorophyll/env/lib/python3.11/site-packages/pyperclip/__init__.py", line 659, in lazy_load_stub_copy
    return copy(text)
           ^^^^^^^^^^
  File "/run/host/tmp/chlorophyll/env/lib/python3.11/site-packages/pyperclip/__init__.py", line 336, in __call__
    raise PyperclipException(EXCEPT_MSG)
pyperclip.PyperclipException:
    Pyperclip could not find a copy/paste mechanism for your system.
    For more information, please visit https://pyperclip.readthedocs.io/en/latest/index.html#not-implemented-error


After the constant propagation optimizer runs, the temps should be removed from the symbol table

As title

This affects t100 codegen because all temps are globals (!)

Related to issue #219


Add Visitors Geolocation Map

GoAccess currently displays visitor city and country under Visitor Hostnames and IPs. Creating a new module with map displaying location of visitors will be a great enhancement.
This can be broken down into two types of maps. Current Visitors similar to Google's Analytics Real Time and Historical Visitors.

I can look into this. However, I wonder if you need a real-time geolocation map considering that GoAccess displays all its panels in real-time (as opposed to Google Analytics where the majority of the data is not real-time)... Thoughts?

By "real time" what is meant is limiting it to the past 20 minutes or 1 hour.

Got it. I actually plan to add the ability to filter complete datasets, issue #117 should handle time/date ranges in real time.
Thanks for the suggestion! Please keep this open so I can look into it.

Hi, I just started using GoAccess two days ago and it is a wonderful tool, amazing work.
Is there anything new/planned about this feature ?

@Natsec Certainly there's still a plan to add this, just haven't got the chance to work on this specific issue. The plan is to implement a d3.js map and probably with the option to have the bar/line charts in addition to the map.

Is it implemented yet? I can’t seem to find it

@AcidRain9 unforgettably it hasn't. stay tuned!

Hi,
Any hopes that it will be implemented?
Thanks for developing this great tool!

@floviolleau It will. Looking into it. Stay tuned.

Hy, any news on that?
It would be awesome to have a city level map like this (circles in one colour)

[img src: https://www.zeemaps.com/blog/radius-map/]

@samularity, alright, so I was just checking out how to make this work, and it doesn't seem too complicated, at least for a basic version. We probably need to add a new GeoJSON file to the report and tweak the current charts a bit. Do you know your way around JS and D3?

@samularity, Do you know your way around JS and D3?
unfortunately not

If I can't add it in an early release, I'll include it in v2.0. Stay tuned for updates.

This feature has been added and it will be included in the upcoming release. Stay tuned You'll still have the choices for area and bar charts available.
Don't hesitate to create a new issue if you encounter any problems with this. Thanks again!


[B8]About making Alumentum

About Alumentum, this issue only appears in B8.

This should be fixed in B9


Update installation-configuration.md

Fixes #118.

@jaxxstorm I think we need this to fix the link on this page:
https://www.pulumi.com/registry/packages/scaleway/installation-configuration/
Look ok to you?


setting a react starter project with mind-ar-js

Hi @hiukim, I challenged myself to use your library in a React project. I used jsDeliver to host the library on a CDN. And then created a component with the source code of example 1 from this repo. Unfortunately, I didn't succeed and got the following error:

Any tips to tackle this? 🙂
You can find my repo here.

@hiukim Apart from an npm package, the only other way I know to include an external library is through a CDN. That approach didn't work with the development build as mentioned in the previous comment. If you've any other workaround, I'm willing to give that a try.
Also, are you active on LinkedIn? I'll tag you in my next post about a demo I created with this library.

@devhims Thanks for reaching out. Just added you in LinkedIn.
I will probably publish an npm package soon. and allow it to be used in React also seems important. So if you are not in particular hurry, I can probably work on that.
I was occupied writing up the documentation before. But it's done now! (do check it out :), so I'll probably have some time on this matter.

Awesome work on the documentation @hiukim! I really like your writing because it feels less mechanical and more human. I noticed the same while going through the README. :)
One reason, I got excited about your library is that I'm also working on a React project which simplifies 3d view and AR with model viewer web component. Currently, image tracking is not available with model viewer and is also not in their pipeline, so I was looking for an alternative. Between Ar.js and your library, I found your README instructions much easier to get started with and the demos seemed quite promising. And I must admit, the idea of an independent developer creating useful tools for others is massively inspiring to me.
Once you make the NPM package, I'll add examples for Image tracking based on your library. There is no rush, you can take your time. ✌

haha. thx.  Yes, let's all show our respects to Linus.
Yeah, I'd definitely love to see more works based on this library. Please share with us whenever you have something. :)

Sure, I'll tag you when I post the demo on LinkedIn. Btw who is Linus?

I mean Linus Torvalds, haha.

just published a npm package. https://www.npmjs.com/package/mind-ar
Also created a React and npm example here: https://github.com/hiukim/mind-ar-js-react

just published a npm package. https://www.npmjs.com/package/mind-ar
Also created a React and npm example here: https://github.com/hiukim/mind-ar-js-react

Super @hiukim !! Really awesome!💯💥🎉


Remove the selectable feature

Codecov Report

Merging #214 into develop will decrease coverage by <.01%.
The diff coverage is 100%.

@@             Coverage Diff             @@
##           develop     #214      +/-   ##
===========================================
- Coverage    91.75%   91.75%   -0.01%
===========================================
  Files           48       48
  Lines         1699     1698       -1
  Branches       146      146
===========================================
- Hits          1559     1558       -1
  Misses         119      119
  Partials        21       21

Impacted Files
Coverage Δ

datahub/core/serializers.py
91.48% <ø> (-0.18%)
:arrow_down:

datahub/metadata/admin.py
100% <100%> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 85f066a...a6190b4. Read the comment docs.

The front end uses selectable when adding interactions (for interaction type) so we need to kill that first.


add rgp lua docs to main repo

Adds rgplua documentation to docs.

The images aren't working, and the sidebar menu needs some work. Also, whatever translates text to the  tag is leaving html codes in the text. But it's a start.

Docs site should be working now.
Couple things with how the docs site now parses and displays the docs:

All images should go in a subfolder assets. An assets folder can be created anywhere inside the top-level docs folder in this repo. You can reference the images with relative paths as you'd expect (and did in your PR)
imgs folder is renamed to assets as the site now supports static files of any type. For instance, you can also include files to download inside an assets folder in the docs.
The name in the sidebar is based on the file name. So docs/rgp-lua/getting-started becomes "Getting Started". Words are split by - or _, and the beginnings of each word is capitalized (with two exceptions for "RGP Lua" and "JW Lua" so those are displayed correctly).

If a section does not have a "main" file (see next bullet), the top-level file for that section will just include the outline for that section. See finalelua.com/docs/rgp-lua for an example.
A section can include a "main" file. This is simply a markdown file in the docs folder with the same name as the section. For instance, docs/getting-started/ has a "main" file of docs/getting-started.md.

Hopefully this makes sense. With #127, I've updated the docs from this PR to work with the docs site.


Add custom graph nodes to graph shader editor using plugins.

I think it would be useful if the shader editor in godot could have some premade nodes like a panner to allow quickly to do animated textures (like river flowing, lava etc).
The input would be:

uv map (the uv map of the object)
x (the intensity of the animation in u coordinate)
y (the intensity of the animation in v coordinate)
time

and the output would be an animated uv coordinate
I made a screenshot explaining the idea

Thinking in a generic way, it would be very useful to be able to create node group and being able to export them. Which means even if we wouldn't have the panner we could easily create our own groups and sharing them.

Sounds like great feature for 2.1 - which will be focused on easy plugin creation

By design rule, Godot shouldn't implement project specific features, instead some kind of custom graph node might be implemented.

I agree but this feature (panner or whaterever is included) isn't specific to a project. It's used in many shaders (like waterfalls, clouds, convoyer belts, etc). It's only a brick that allows to quickly design a cool material and it's present in many engines (like Unreal or Unity).

Yeah, i recently realized hoy to do it by jus inherit resource, so plugins wont be involved at all. Anyway the editor will most likely be rewritten for 2.2 and/or 3.0

Shader node groups would be awesome and if the interface would work similar to how blender node groups work then that would be epic, especially the way you can instance those groups inside different materials.
It also means those of us willing and capable of building more complex shader/nodegraphs can share our node snippets with those who might not, thus making their lives easier too. :)

Moving to the next milestone as 3.1 is now feature frozen.
Also, is it still relevant in the current master branch with new VisualShader?

Far as I can tell, still as relevant as ever.