New Authenticate Middleware - multiple guards

Laravel Version: 5.7.11
PHP Version: 7.2.10

Description:
L5.7 includes new middleware to handle and redirect unauthenticated users.
It works well with "web" guard. But using custom guards there is no way (at least I couldn't find it) to get current guard.
Different guard would redirect authenticated users to different route, but guard is always set to "web" instead of current guard.
Steps To Reproduce:
in auth.php - new guard is added: admin
Routes:
Route::group(['middleware' => ['auth:admin']], function () {
    // routes...
});

$this->auth returns AuthManager with protected property $guards: [admin];
but $this->auth->guard() returns "web".
Shouldn't $this->auth->guard() return "admin", instead of "web" SessionGuard, since this route is under middleware auth:admin?

I have same issue, right now i am handling multiple guards like this
<?php

namespace App\Http\Middleware;

use Illuminate\Auth\Middleware\Authenticate as Middleware;
use Illuminate\Support\Facades\Route;

class Authenticate extends Middleware
{
    /**
     * Get the path the user should be redirected to when they are not authenticated.
     *
     * @param  \Illuminate\Http\Request $request
     * @return string
     */
    protected function redirectTo($request)
    {
        if (!$request->expectsJson()) {
            if (Route::is('admin.*')) {
                return route('admin.login');
            }
            return route('login');
        }
    }
}

There should be a way to receive the guard name in redirectTo method.

I did override on unauthenticated method, inside app/Exceptions/Handler.php:
protected function unauthenticated($request, AuthenticationException $exception)
{
    if ($request->expectsJson()) {
        return response()->json(['message' => $exception->getMessage()], 401);
    }

    $guard = array_get($exception->guards(), 0);

    switch ($guard) {
        case 'admin':
            $route = 'admin.login';
            break;
       default:
            $route = 'login';
    }

    return redirect()->guest(route($route));
}

In the class "Authenticate" I just override function "authenticate" and changed function "redirectTo" like this(my custom guard is "admin"):
auth->guard($guard)->check()) {
				return $this->auth->shouldUse($guard);
			}
		}

		$guard = $guards[0];

		if ($guard == 'admin'){
			$request->path = 'admin.';
		}else{
			$request->path = '';
		}

		throw new AuthenticationException(
			'Unauthenticated.', $guards, $this->redirectTo($request)
		);
	}

    protected function redirectTo($request)
    {
        return route($request->path . 'login');
    }
}

Think we can add this. Just pass along the guards argument to the redirectTo method. Feel free to send in a PR to master.

As said in the linked PRs you could easily solve this by adding a property to your middleware. So gonna close this.

In the class "Authenticate" I just override function "authenticate" and changed function "redirectTo" like this(my custom guard is "admin"):
<?php

namespace App\Http\Middleware;

use Illuminate\Auth\AuthenticationException;
use Illuminate\Auth\Middleware\Authenticate as Middleware;

class Authenticate extends Middleware
{

	protected function authenticate($request, array $guards)
	{
		if (empty($guards)) {
			$guards = [null];
		}

		foreach ($guards as $guard) {
			if ($this->auth->guard($guard)->check()) {
				return $this->auth->shouldUse($guard);
			}
		}

		$guard = $guards[0];

		if ($guard == 'admin'){
			$request->path = 'admin.';
		}else{
			$request->path = '';
		}

		throw new AuthenticationException(
			'Unauthenticated.', $guards, $this->redirectTo($request)
		);
	}

    protected function redirectTo($request)
    {
        return route($request->path . 'login');
    }
}

this code still gives this error
Symfony \ Component \ Debug \ Exception \ FatalThrowableError (E_ERROR)
Call to undefined method Illuminate\Auth\TokenGuard::attempt()

handler
just edit
$guard = $exception->guards()[0];

How can we fix this problem in Laravel 7

I have same issue, right now i am handling multiple guards like this
<?php

namespace App\Http\Middleware;

use Illuminate\Auth\Middleware\Authenticate as Middleware;
use Illuminate\Support\Facades\Route;

class Authenticate extends Middleware
{
    /**
     * Get the path the user should be redirected to when they are not authenticated.
     *
     * @param  \Illuminate\Http\Request $request
     * @return string
     */
    protected function redirectTo($request)
    {
        if (!$request->expectsJson()) {
            if (Route::is('admin.*')) {
                return route('admin.login');
            }
            return route('login');
        }
    }
}

There should be a way to receive the guard name in redirectTo method.
PS
I was able to solve this, see how
<?php

namespace App\Http\Middleware;

use Closure;
use Illuminate\Auth\Middleware\Authenticate as Middleware;

class Authenticate extends Middleware
{
    /**
     * @var array
     */
    protected $guards = [];

    /**
     * Handle an incoming request.
     *
     * @param  \Illuminate\Http\Request $request
     * @param  \Closure $next
     * @param  string[] ...$guards
     * @return mixed
     *
     * @throws \Illuminate\Auth\AuthenticationException
     */
    public function handle($request, Closure $next, ...$guards)
    {
        $this->guards = $guards;

        return parent::handle($request, $next, ...$guards);
    }

    /**
     * Get the path the user should be redirected to when they are not authenticated.
     *
     * @param  \Illuminate\Http\Request $request
     * @return string
     */
    protected function redirectTo($request)
    {
        if (!$request->expectsJson()) {
            if (array_first($this->guards) === 'admin') {
                return route('admin.login');
            }
            return route('login');
        }
    }
}

Thanks. It works. But In laravel ^7.0, array_first() does not work. It is now Arr:first().

I have same issue, right now i am handling multiple guards like this
<?php

namespace App\Http\Middleware;

use Illuminate\Auth\Middleware\Authenticate as Middleware;
use Illuminate\Support\Facades\Route;

class Authenticate extends Middleware
{
    /**
     * Get the path the user should be redirected to when they are not authenticated.
     *
     * @param  \Illuminate\Http\Request $request
     * @return string
     */
    protected function redirectTo($request)
    {
        if (!$request->expectsJson()) {
            if (Route::is('admin.*')) {
                return route('admin.login');
            }
            return route('login');
        }
    }
}

There should be a way to receive the guard name in redirectTo method.
PS
I was able to solve this, see how
<?php

namespace App\Http\Middleware;

use Closure;
use Illuminate\Auth\Middleware\Authenticate as Middleware;

class Authenticate extends Middleware
{
    /**
     * @var array
     */
    protected $guards = [];

    /**
     * Handle an incoming request.
     *
     * @param  \Illuminate\Http\Request $request
     * @param  \Closure $next
     * @param  string[] ...$guards
     * @return mixed
     *
     * @throws \Illuminate\Auth\AuthenticationException
     */
    public function handle($request, Closure $next, ...$guards)
    {
        $this->guards = $guards;

        return parent::handle($request, $next, ...$guards);
    }

    /**
     * Get the path the user should be redirected to when they are not authenticated.
     *
     * @param  \Illuminate\Http\Request $request
     * @return string
     */
    protected function redirectTo($request)
    {
        if (!$request->expectsJson()) {
            if (array_first($this->guards) === 'admin') {
                return route('admin.login');
            }
            return route('login');
        }
    }
}

Thanks. It works. But In laravel ^7.0, array_first() does not work. It is now Arr:first().

It should be Arr::first() not Arr:first().
Just tried Arr:first() and came up with issues.
Also do not forget to add
use Illuminate\Support\Arr; at the beginning of the file
It worked for me this way.

I have same issue, right now i am handling multiple guards like this
<?php

namespace App\Http\Middleware;

use Illuminate\Auth\Middleware\Authenticate as Middleware;
use Illuminate\Support\Facades\Route;

class Authenticate extends Middleware
{
    /**
     * Get the path the user should be redirected to when they are not authenticated.
     *
     * @param  \Illuminate\Http\Request $request
     * @return string
     */
    protected function redirectTo($request)
    {
        if (!$request->expectsJson()) {
            if (Route::is('admin.*')) {
                return route('admin.login');
            }
            return route('login');
        }
    }
}

There should be a way to receive the guard name in redirectTo method.
PS
I was able to solve this, see how
<?php

namespace App\Http\Middleware;

use Closure;
use Illuminate\Auth\Middleware\Authenticate as Middleware;

class Authenticate extends Middleware
{
    /**
     * @var array
     */
    protected $guards = [];

    /**
     * Handle an incoming request.
     *
     * @param  \Illuminate\Http\Request $request
     * @param  \Closure $next
     * @param  string[] ...$guards
     * @return mixed
     *
     * @throws \Illuminate\Auth\AuthenticationException
     */
    public function handle($request, Closure $next, ...$guards)
    {
        $this->guards = $guards;

        return parent::handle($request, $next, ...$guards);
    }

    /**
     * Get the path the user should be redirected to when they are not authenticated.
     *
     * @param  \Illuminate\Http\Request $request
     * @return string
     */
    protected function redirectTo($request)
    {
        if (!$request->expectsJson()) {
            if (array_first($this->guards) === 'admin') {
                return route('admin.login');
            }
            return route('login');
        }
    }
}

Thanks. It works. But In laravel ^7.0, array_first() does not work. It is now Arr:first().

It should be Arr::first() not Arr:first().
Just tried Arr:first() and came up with issues.
Also do not forget to add
use Illuminate\Support\Arr; at the beginning of the file
It worked for me this way.

This worked fine in Laravel 8 :)
$guard = Arr::first( $this->guards );
        switch ( $guard ) {
            case 'supplier':
                return route( 'supplier.login.supplier' );
                break;
        }


Provide type safety for useStorage

Describe the feature
The current useStorage intellisense only defined it can either be a string or undefined. It would be nice to extend it to allow for autocomplete suggestions when using the storage while still allowing undefined, to improve the DX.
Additional information

[X] Would you be willing to help implement this feature?

https://github.com/unjs/nitro/pull/1996#issuecomment-2034557502


FR: Add "ignore note" option in Hierarchy notes

With the addition of Hierarchy notes I'm missing the chance of specifying files ot be ignored. Case in point: templates.

I'd love to be able to hide these, as any modification withing the template would affect any rendered file, the hierarchy notes are perfect for this.

I don't fully understand...
So do you have a Hierarchy Note Template?
And the templates in that note are being added to your Breadcrumbs?

My bad. This is not my first language and it shows. I'm using templates to render new notes, and these templates are being listed in the matrix view as well.
I can't find a way of hiding trem without modifying the template, which requires to later manually modify every new note.
I'm asking whener it would be possible to list in a Hierarchy Note certain notes that should be ignored by the system.

I've added a fix for this, but it may not be what you meant.
Can you send an example of a note where this happens?

Oh my, my bad. There is no bug! I'll try to explain myself.
There is currently no way to prevent breadcrumbs from listing notes.
Because of this, templates with the yaml properly defined are listed among
rendered notes.
To solve this, we can't write anything on templates, because any note
rendered with them would inherit the unlisting.
I proposed using Hierarchy notes for this, as it fits to have written down
which files to ignore next to which files to index, but that's all. I was
asking for a method to unlist notes, sorry for all this confusion :/
El mar., 14 dic. 2021 6:50, SkepticMystic @.***>
escribió:

I've added a fix for this, but it may not be what you meant.
Can you send an example of a note where this happens?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/SkepticMystic/breadcrumbs/issues/207#issuecomment-993181392,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AR6XMLMT5O3U6S37YQA4UKDUQ3LJ3ANCNFSM5JSBR23A
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

@veradrawer hahaha ok, I think I understand better...
Can you send an example of a template that causes this problem

Sure, daily notes are a prime example:

Which are rendered with:
---
modified:
created: "<% tp.date.now("YYYY-MM-DD HH:mm") %>"
title: "<% tp.file.title %> - "
aliases:
notetype: [["_fleeting"]]
noteclass:
- [["_daily"]]
- [["_event"]]
day: [["<% tp.date.now("YYYY-MM-DD") %>"]]
month: [["<% tp.date.now("YYYY-MM-MMM") %>"]]
quarter: [["<% tp.date.now("YYYY-[Q]Q") %>"]]
year: [["<% tp.date.now("YYYY") %>"]]
related:
parent: [["_daily"]]
next: [["<% tp.date.now("YYYY-MM-DD", 1) %>"]]
prev: [["<% tp.date.now("YYYY-MM-DD", -1) %>"]]
tags:
---

This also happens, for example, with Literary notes:

Rendered from:
---
modified:
created: "{{date:YYYY-MM-DD}} {{time:HH:mm}}"
title: "{{title}}"
aliases: "{{citekey}}"
citekey: "{{citekey}}"
notetype: [["_literature"]]
noteclass:
- [["_zotero"]]
author: {{authorString}}
publisher: [["{{publisher}}"]]
collection:
lang:
year: [["{{year}}"]]
quarter:
period:
docType:
mediaformat:
- [["_analogical"]]
- [["_digital"]]
url: "{{URL}}"
related:
parent: [["_references"]]
tags:
---

@veradrawer I've been thinking about this, and to tell Breadcrumbs to ignore a field, I see the following options:

Put something in the key of the field: !parent:: [[note]]. This has the issue that now it's a different field, you know? Dataview and Obsidian see that as completely different from parent.
Put something in the value of the field: parent: ! [[Note]]. But this won't work without a big refactoring, because the current system just grabs all wikilinks inside the fields you specify in your hierarchies.
Put something inside the wikilink: parent:: [[!Note]]. This has the issue that Breadcrumbs has to do extra parsing of every breadcrumb in the vault. And there might be edge cases where a link has the ignore character ! but you still want it to be included.

So, the way I see it, no matter what, you're gonna have to add an ignore character somewhere. So my suggestion is to add a backslash before the field key, making Breadcrumbs think it's a different field entirely \parent:: [[Note]], and therefore will ignore it. It requires you to manually remove it after inserting the template, but I don't see a better way right now.
What are your thoughts on this?

I believe #306 closes this


feature request: middleware and software packs

It seems like the parser doesn't care about middlewares(freertos, usb_device, etc.). As this project is a "minimal" ioc parser, is it OK to take care of these additional components?
Or alternatively, can we add a .cproject parser, and directly extract sources/include paths from CubeIDE project file?

see also caveats, 4th point. (with example how to add such things manually).
this project is supposed to be simple / lightweight, so at the moment I consider this out of scope.


[Feature]: Additional string codecs

Some strings are prefixed with some kind of length byte(s).
This is a generic class which by default uses a u8 as length prefix but can use any Unsized<number> as prefix codec
No tests have been made or done. But it is relatively straight forward

This pr also changes FixedLength to be FixedLengthString to be consistent with other strings like CString and PrefixedString


Limit radius in which objects are considered to alignment

If I have a very large diagram with many objects, it seems like draw.io is considering every object in the diagram for alignment of the current object (i.e. the blue guides may appear to align the current object with one that is perhaps 20 or 30 elements away, so it not even in view on my screen). It would be useful to limit the objects considered for alignment to some local area around the current object, since most often I need to align the object locally to some nearby object, rather than one that is very far away in the diagram.

Tested in Chrome Version 74.0.3729.131 (Official Build) (64-bit)
In the test case, 6 shapes were considered for alignment. When it comes to large diagrams, I assume that number goes higher which can hinder working

Given the age of this issue and it having a single vote I'm going to decline it due to lack of interest. Thank you for taking the time to make the suggestion.


[docs] Flatten quickstart directory into quickstart.yaml file

@timothysc PTAL
Since we removed the component YAML files for the Quickstart, it's probably more clear to have examples/quickstart.yaml rather than examples/quickstart/aggregate.yaml (where aggregate.yaml was the only file in the directory).
This PR makes that change and updates the Makefile and other documentation as necessary.
Signed-off-by: Jessica Yao<EMAIL_ADDRESS>

Dunno why we had a link to the Dockerfile but it didn't make sense in that particular context anyways ("environmental variables") -- removed. This should be good to merge


Release PR-01-23

Proposed changes

fixed issue with price change representation;
fixed issue with broken links on dashboards.

Types of changes

[x] Bugfix
[ ] New feature or Enhancement
[ ] Others

Checklist

[x] I have read the CONTRIBUTING GUIDELINES doc
[x] I have signed the CLA by posting "I have read the CLA Document and I hereby sign the CLA" in the comment
[ ] Lint and unit tests pass locally with my changes
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] I have added necessary documentation (if appropriate)
[ ] Any dependent changes have been merged and published in downstream modules

Accept

@Victorius  could you sign like this
I have read the CLA Document and I hereby sign the CLA

I have read the CLA Document and I hereby sign the CLA

recheck

I have read the CLA Document and I hereby sign the CLA

I have read the CLA Document and I hereby sign the CLA


Test upstream PR fix for flaky tests

Codecov Report

Merging #1106 (67d95a9) into main (18890ec) will increase coverage by 0.16%.
The diff coverage is n/a.

:mega: This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more
@@            Coverage Diff             @@
##             main    #1106      +/-   ##
==========================================
+ Coverage   62.55%   62.72%   +0.16%
==========================================
  Files          72       72
  Lines        9439     9439
==========================================
+ Hits         5905     5921      +16
+ Misses       3128     3115      -13
+ Partials      406      403       -3

Impacted Files
Coverage Δ

internal/provider/kubernetes/controller.go
51.24% <0.00%> (+0.67%)
:arrow_up:

internal/provider/kubernetes/helpers.go
85.13% <0.00%> (+6.75%)
:arrow_up:

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


Reseting a promise?

I've come across the need to be able to reset a promise. That is, if the promise is resolved, I want to mark the promise as unresolved, then re-trigger the original call that created that promise, and have this re-trigger all the chain of then()s that are attached to it.
It looks like promises are a one time object, and once resolved they get rid of all the attached callbacks and stay resolved forever. So the most obvious way to re-trigger the behavior is to throw away the old promise, create a new one, and attach all the then()s again. However this does not work when you have multiple modules that use the promise to attach additional then()s on it. Since the module where the promise is created has no knowledge of how many and how long are the then chains that have been attached, it cannot re-attach everything by itself.
This need has occurred in client-side applications where the promise objects are shared around across multiple modules that make use of the fetched data, often attaching then() functions that do some further transformation. It would be very useful if I could persist a single promise object in memory for a long time, and be able to reset() it at the data layer without having to write additional code for all clients of that data layer so that they re-trigger (or re-attach) the then()s to a new promise everytime the data is reloaded.

You're fundamentally failing to understand how promises work. You're basically looking at a screw and asking for it to be changed into a Hammer because you tried to screw in a nail.
Promises will never be resettable. That violates one the invariants - that a promise is only resolved once.
 Node.js/io.js has a Stream based on an Event Emitter, and it should work for what you want. I've yet to see an abstraction for pipelines in a similar vein of Promises being an abstraction for an asynchronous return value. The Stream code is itself "pure", and there are other Stream implementations too, for client side code. Plus browserify.

@dtheodor I believe what you are looking for is a Signal. I wrote up a long essay about finding the right tool for the job which might be insightful. https://github.com/kriskowal/gtor
@havvy Thanks for your feedback. I find that people feel more welcome if you open by explaining how you understand their motivation.

Yeah, I'm not always very good at putting sympathy first. It's my natural state, but I forget that it's not visible to others. Your link is interesting, and I'm going to read it.

Hey thanks for the feedback.
I had looked both at signals and reactive libraries (streams). The thing is that the Promise API is very close to what I need, because it combines

chaining of post-processing (reactive does this too, and better)
information about whether a request is ongoing (unresolved promise). I could not find any concept of this in a reactive library (maybe I did not look good enough)
the API itself is exactly the API I need to consume, unlike signals or streams where more work needs to be done

I know my use-case can be implemented with signals, which I did.
However I opened the issue to ask why can a promise not being reset (and in case I missed the hidden reset API somewhere). Is this an argument against mutability?
None of the following statements answer the why.

You're fundamentally failing to understand how promises work. You're basically looking at a screw > and asking for it to be changed into a Hammer because you tried to screw in a nail.
Promises will never be resettable. That violates one the invariants - that a promise is only resolved > once.

A promise is like an async return value. I.e., synchronous functions return values, asynchronous functions return promises.
The caller of a synchronous function can't change the return value of that function. Similarly, the caller of an asynchronous function can't change the return value of that function, i.e., nobody can reset a promise.

Ok I can see that argument. You treat a promise as an immutable future value, while I wanted to use it more like an event to chain callbacks on.

@dtheodor This may be useful for your usecase: http://highlandjs.org/

Thanks for the discussion.


Make Tree view sidebar resizable

Thanks for making this! This PR makes it possible to adjust Tree view sidebar width and persist the value into Nightlight's config.

Looks great. A few things:

It seems that there is a typo in the CSS, as it is broken for me. I think .leftsidebar needs an end bracket.
After fixing the CSS it works for me, but on windows it the line isn't really visible because it is sitting flush with the scroll bar:

When I dragged it all the way to the left (so the left-sidebar-width was 0) I was unable to drag it anymore. Probably has to do with the windows scrollbar getting in the way.

Maybe there's another way to display a resizer control? When I get time i'll try to think of one...

Thanks for taking a look. I’ll update the PR with fixes.

Added hide/show buttons to deal with 3rd issue, also fixed CSS, sorry it was reformatted in my editor :(

Thanks, this is a good solution. I added some styling fixes and prevented the toolbar from moving when resizing (it was overlapping with the settings). Let me know if you notice any problems due to my changes. After i get a chance to try it more i'll push a release.

I was about to make InstaREPL resizable as well, what do you think?

Yeah that sounds great. Since it can be toggled off, we should probably have a minimum size. Otherwise it could get into a state where toggling on/off doesn't appear to do anything.

Had to temporarily move the resizer stuff to https://github.com/oakes/Nightlight/tree/resizer because it looks like things are broken during advanced compilation. Might be something I did, I'm not sure. I'll look into it when i get the chance

I released version 2.2.2 with sidebar resizing included. Turns out that the broken compilation only seems to happen on Windows, so I just compiled the release in WSL and it seems to work fine.


Wrapper function/type for creating Scalars

          Could add a helper for creating the static somewhere, seen it in a few instances now

Originally posted by @jordanrfrazier in https://github.com/kaskada-ai/kaskada/pull/784#discussion_r1342918969

Arrow recently added Scalar which simplifies many of the APIs handling scalar values. We could potentially use this to replace our ScalarValue enum.
See https://github.com/apache/arrow-rs/pull/4793/files for some notes:

We can use Int64Array::new_scalar(...) to simplify creating a scalar of a specific type.
We can create a Scalar<ArrayRef> for using an array as a scalar.

This should let us create something like:
struct ScalarValue(Scalar<ArrayRef>);

// 1. Serde the underlying ArrayRef
// 2. Support use as a scalar anywhere

@jordanrfrazier FYI -- I think we should be able to completely replace our ScalarValue enum with Scalar<ArrayRef>. I think it's probably worth doing that for how we serialize scalars in the physical plan.
Thoughts on:
A) Trying to change the ScalarValue to use that representation (potentially messy since we have that protobuf in many places0.
B) Introducing a parallel Scalar??? to represent that?
C) Just using arrow_array::Scalar<arrow_array::ArrayRef> in those places?
I think with C we'd need to do the custom serialization like we did for structs with an ArrayRef, while with A or B we could encapsulate that in our serialization of the wrapper type we create.


Qwik construct

Requests:

kujtimii https://discord.com/channels/@me/992051892148502528/1062354171698876438

@fwang Any ETA on this?

We are transition this repo to https://github.com/sst/v2 and are closing this issue for now. If it is still relevant please reopen it there - sorry for the trouble


Skip captcha for e2e tests

We want to add e2e tests with cypress to our Stripe application.
We need to skip the captcha in testing account.
So, we will able to login to Stripe dashboard as part of the e2e test.

Hi there,
Could you please provide a screen-recording or a screen-shot of the cypress test run that fails due to the captcha? This will provide us some additional context we need to look into the issue further.
Thanks!

Hi,
We are trying to simulate the login flow in cypress.
After typing the email and password and click 'Continue' we get this captcha:


Better alerts

Right now the only indication the timer finishes is a temporary "toast" style alert. This isn't great if you're focused on something, as you're likely to miss it.
Some ideas:

Have a small chime play. Should be able to turn off or on (likely possible, but may be annoying)
Have the panel do some kind of flash/animation to visually indicate it's done (likely built off of work for #1)
Have the SilverBullet task icon (when using as a PWA) flash or change in some fashion (may not be possible with current API)

Supporting alert sound with 7cb4e29
Leaving this open, as their should also be a more prominent visual component


Allow merger to publish head_block_drift promethus metrics when it is far behind live blocks

If merger is merging blocks from 1 year ago, it doesn't publish head_block_drift promethus metrics. It only publishes this data when closer to "live". This would be nice to have.

This seems not to be working.  I still see no data.  Blocks are being merged by mindreader... merger is just reading them in as they are created.
This returns nothing:
curl -s http://localhost:9102 | grep drift

you're describing a different scenario than the original issue (mindreader is merging vs merger is merging old blocks -- there was a case where it wouldn't update the head block drift, now fixed here --).

ok, well multiple issues. ;-)
Here is dfuse drift chart from the time the chain switches from catching up to "live" for the first time.  (the first mindreader caught up)


Token factions should considere visibility status

If I give a color to tokens according to factions like friendly, enemy etc. token faction works great and enhances the overview I and the players have when a lot of icons are on the game board.
As a GM though I want to place new tokens on the board (ALT and draw an icon on the board) or I might make a token invisible when they lay in ambush. As a GM token factions helps to keep those token organized but as a player I should not be able to see those invisible icons. Token faction keeps drawing a faction color around those invisble tokens and thus reveals them.
A new setting "Show invisible faction colors only to GM"  would be a nice improvement.

Not sure if this is similar or not, but you can see the Token Factions through fog of war and when characters are not in line of sight. I couldn't find a setting or anything. Hope this could be added. Would be awesome to not show the factions unless the token itself is visible.

I have the same issue. It used to only show the border for visible tokens but now all of them are visible. Border Control (which does a similar thing) also causes the same issue.

The problem is fvtt10....and the new logic he implemented I was able to handle the use cases on the player side but unfortunately the GM when he checks a token will see the faction designs of the tokens.

Even with this "problem" now the module should be usable, unfortunately I am not a PIXI expert.

Thanks for the update. This does fix all factions visible to players but unfortunately it doesn't display a faction until the player hovers over the token (except for their owned tokens):

Should be finally fixed 0.3.8. Try out and let me know it.

Should be finally fixed 0.3.8. Try out and let me know it.

It's working perfectly! We happened to have to cancel our last session where I was gonna use this due to a player being sick. So I'm super excited to be able to use it for our upcoming session.
Thanks so much for a great module!


feat: Add python model support

Description
Support dbt python models by using Spark
Docs - https://docs.aws.amazon.com/athena/latest/ug/notebooks-spark.html
Bugs currently identified:

mock_athena does not support these functions yet
Incremental model does not fully utilize spark capabilities

Prerequisites

A spark enabled work group created in athena
Spark execution role granted access to Athena, Glue and S3
The spark work group is added to the ~/.dbt/profiles.yml file and the profile is referenced in dbt_project.yml

analytics_spark:
  outputs:
    dev:
      database: awsdatacatalog
      region_name: us-east-1
      s3_data_dir: s3://dbt-athena/
      s3_staging_dir: s3://<user>-athena-query-results/
      schema: analytics_dev
      threads: 4
      type: athena
      work_group: primary
      spark_work_group: spark
  target: dev

Models used to test - Optional

You can add the below models to your dbt project or you can clone this repository and do poetry installation & run
model
{{ config(materialized="table") }}
select 1 as column_1, 2 as column_2, '{{ run_started_at.strftime("%Y-%m-%d") }}' as run_date

python_table
import pandas as pd

def model(dbt, session):
    dbt.config(materialized="table")

    model_df = pd.DataFrame({"A": [1, 2, 3, 4]})

    return model_df

python_incremental
import pandas as pd

def model(dbt, session):
    dbt.config(materialized="incremental")
    df = dbt.ref("model")

    if dbt.is_incremental:
        max_from_this = f"select max(run_date) from {dbt.this}"
        df = df.filter(df.run_date >= session.sql(max_from_this).collect()[0][0])

    return df

Build
dbt output

============================== 2023-04-03 23:24:42.103241 | 3d6bdb09-bba6-4b8c-9bdc-ff76eb1afb75 ==============================
23:24:42.103241 [info ] [MainThread]: Running with dbt=1.4.4
23:24:42.105171 [debug] [MainThread]: running dbt with arguments {'debug': True, 'write_json': True, 'use_colors': True, 'printer_width': 80, 'version_check': True, 'partial_parse': True, 'static_parser': True, 'profiles_dir': '/home/avinash1394/.dbt', 'send_anonymous_usage_stats': True, 'quiet': False, 'no_print': False, 'cache_selected_only': False, 'which': 'run', 'rpc_method': 'run', 'indirect_selection': 'eager'}
23:24:42.105507 [debug] [MainThread]: Tracking: tracking
23:24:42.106708 [debug] [MainThread]: Sending event: {'category': 'dbt', 'action': 'invocation', 'label': 'start', 'context': [<snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x7efecc7e7d30>, <snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x7efeca585c60>, <snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x7efeca5847c0>]}
23:24:42.115289 [debug] [MainThread]: checksum: 170d819e8a7f11e09c497566dd7f61e1355cb9fb514921503937b951cb4a2250, vars: {}, profile: None, target: None, version: 1.4.4
23:24:42.135156 [debug] [MainThread]: Partial parsing enabled: 0 files deleted, 0 files added, 0 files changed.
23:24:42.135605 [debug] [MainThread]: Partial parsing enabled, no changes found, skipping parsing
23:24:42.141326 [debug] [MainThread]: Sending event: {'category': 'dbt', 'action': 'load_project', 'label': '3d6bdb09-bba6-4b8c-9bdc-ff76eb1afb75', 'context': [<snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x7efeca5480d0>]}
23:24:42.146448 [debug] [MainThread]: Sending event: {'category': 'dbt', 'action': 'resource_counts', 'label': '3d6bdb09-bba6-4b8c-9bdc-ff76eb1afb75', 'context': [<snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x7efeca518490>]}
23:24:42.146964 [info ] [MainThread]: Found 3 models, 0 tests, 0 snapshots, 0 analyses, 313 macros, 0 operations, 0 seed files, 0 sources, 0 exposures, 0 metrics
23:24:42.147303 [debug] [MainThread]: Sending event: {'category': 'dbt', 'action': 'runnable_timing', 'label': '3d6bdb09-bba6-4b8c-9bdc-ff76eb1afb75', 'context': [<snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x7efecc7e7d30>]}
23:24:42.148463 [info ] [MainThread]:
23:24:42.150216 [debug] [MainThread]: Acquiring new athena connection 'master'
23:24:42.151734 [debug] [ThreadPool]: Acquiring new athena connection 'list_awsdatacatalog'
23:24:42.152344 [debug] [ThreadPool]: Opening a new connection, currently in state init
23:24:42.723778 [debug] [ThreadPool]: On list_awsdatacatalog: Close
23:24:42.727164 [debug] [ThreadPool]: Acquiring new athena connection 'list_awsdatacatalog_analytics_dev'
23:24:42.730022 [debug] [ThreadPool]: Opening a new connection, currently in state closed
23:24:43.445183 [debug] [ThreadPool]: On list_awsdatacatalog_analytics_dev: Close
23:24:43.450169 [debug] [MainThread]: Sending event: {'category': 'dbt', 'action': 'runnable_timing', 'label': '3d6bdb09-bba6-4b8c-9bdc-ff76eb1afb75', 'context': [<snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x7efeca5c9390>]}
23:24:43.451946 [info ] [MainThread]: Concurrency: 4 threads (target='dev')
23:24:43.452818 [info ] [MainThread]:
23:24:43.464314 [debug] [Thread-1 (]: Began running node model.dbt_athena_spark.model
23:24:43.465632 [debug] [Thread-2 (]: Began running node model.dbt_athena_spark.my_first_dbt_model
23:24:43.466841 [info ] [Thread-1 (]: 1 of 3 START sql table model analytics_dev.model ............................... [RUN]
23:24:43.467984 [info ] [Thread-2 (]: 2 of 3 START python table model analytics_dev.my_first_dbt_model ............... [RUN]
23:24:43.469556 [debug] [Thread-1 (]: Acquiring new athena connection 'model.dbt_athena_spark.model'
23:24:43.471026 [debug] [Thread-2 (]: Acquiring new athena connection 'model.dbt_athena_spark.my_first_dbt_model'
23:24:43.472068 [debug] [Thread-1 (]: Began compiling node model.dbt_athena_spark.model
23:24:43.473049 [debug] [Thread-2 (]: Began compiling node model.dbt_athena_spark.my_first_dbt_model
23:24:43.478703 [debug] [Thread-1 (]: Writing injected SQL for node "model.dbt_athena_spark.model"
23:24:43.517472 [debug] [Thread-2 (]: Writing injected SQL for node "model.dbt_athena_spark.my_first_dbt_model"
23:24:43.518586 [debug] [Thread-1 (]: Timing info for model.dbt_athena_spark.model (compile): 2023-04-03 23:24:43.473840 => 2023-04-03 23:24:43.518438
23:24:43.519504 [debug] [Thread-2 (]: Timing info for model.dbt_athena_spark.my_first_dbt_model (compile): 2023-04-03 23:24:43.479242 => 2023-04-03 23:24:43.519394
23:24:43.520381 [debug] [Thread-1 (]: Began executing node model.dbt_athena_spark.model
23:24:43.521304 [debug] [Thread-2 (]: Began executing node model.dbt_athena_spark.my_first_dbt_model
23:24:43.561076 [debug] [Thread-1 (]: Opening a new connection, currently in state closed
23:24:43.564212 [debug] [Thread-2 (]: Opening a new connection, currently in state init
23:24:44.145523 [debug] [Thread-2 (]: Athena adapter: table_name : my_first_dbt_model
23:24:44.146605 [debug] [Thread-2 (]: Athena adapter: table type : table
23:24:44.161340 [debug] [Thread-1 (]: Athena adapter: table_name : model
23:24:44.162252 [debug] [Thread-1 (]: Athena adapter: table type : table
23:24:44.619308 [debug] [Thread-1 (]: Athena adapter: analytics_dev.model is stored in s3://dbt-athena/analytics_dev/model/1ab22d75-15e7-48f6-bf5c-c7a9d6b33ff9/
23:24:44.645411 [debug] [Thread-2 (]: Athena adapter: analytics_dev.my_first_dbt_model is stored in s3://dbt-athena/analytics_dev/my_first_dbt_model/ab81468d-6efc-48ac-a6f5-1ce660ae758c
23:24:45.158371 [debug] [Thread-1 (]: Athena adapter: Deleting table data: path='s3://dbt-athena/analytics_dev/model/1ab22d75-15e7-48f6-bf5c-c7a9d6b33ff9/', bucket='dbt-athena', prefix='analytics_dev/model/1ab22d75-15e7-48f6-bf5c-c7a9d6b33ff9/'
23:24:45.161924 [debug] [Thread-2 (]: Athena adapter: Deleting table data: path='s3://dbt-athena/analytics_dev/my_first_dbt_model/ab81468d-6efc-48ac-a6f5-1ce660ae758c', bucket='dbt-athena', prefix='analytics_dev/my_first_dbt_model/ab81468d-6efc-48ac-a6f5-1ce660ae758c/'
23:24:45.680401 [debug] [Thread-1 (]: Using athena connection "model.dbt_athena_spark.model"
23:24:45.681385 [debug] [Thread-1 (]: On model.dbt_athena_spark.model: drop table if exists `analytics_dev`.`model`
23:24:45.685620 [debug] [Thread-2 (]: Using athena connection "model.dbt_athena_spark.my_first_dbt_model"
23:24:45.690158 [debug] [Thread-2 (]: On model.dbt_athena_spark.my_first_dbt_model: drop table if exists `analytics_dev`.`my_first_dbt_model`
23:24:47.550615 [debug] [Thread-1 (]: SQL status: OK -1 in 2 seconds
23:24:47.564600 [debug] [Thread-2 (]: SQL status: OK -1 in 2 seconds
23:24:48.314214 [debug] [Thread-1 (]: Athena adapter: S3 path does not exist
23:24:48.315321 [debug] [Thread-1 (]: Writing runtime sql for node "model.dbt_athena_spark.model"
23:24:48.317033 [debug] [Thread-1 (]: Using athena connection "model.dbt_athena_spark.model"
23:24:48.317810 [debug] [Thread-1 (]: On model.dbt_athena_spark.model: -- /* {"app": "dbt", "dbt_version": "1.4.4", "profile_name": "analytics_spark", "target_name": "dev", "node_id": "model.dbt_athena_spark.model"} */

  create table "analytics_dev"."model"
    with (
      table_type='hive',
      is_external=true,
      external_location='s3://dbt-athena/analytics_dev/model/6d1b473e-ee73-4a6c-b9f2-195e54b2b40d',
      format='parquet'
    )
    as

select 1 as column_1, 2 as column_2, '2023-04-03' as run_date

23:24:48.350239 [debug] [Thread-2 (]: Athena adapter: S3 path does not exist
23:24:48.355072 [debug] [Thread-2 (]: Writing runtime python for node "model.dbt_athena_spark.my_first_dbt_model"
23:24:48.356260 [debug] [Thread-2 (]: On model.dbt_athena_spark.my_first_dbt_model:

  import pandas as pd

def model(dbt, session):
    dbt.config(materialized="table")

    model_df = pd.DataFrame({"A": [1, 2, 3, 4]})

    return model_df

# This part is user provided model code
# you will need to copy the next section to run the code
# COMMAND ----------
# this part is dbt logic for get ref work, do not modify

def ref(*args,dbt_load_df_function):
    refs = {}
    key = ".".join(args)
    return dbt_load_df_function(refs[key])

def source(*args, dbt_load_df_function):
    sources = {}
    key = ".".join(args)
    return dbt_load_df_function(sources[key])

config_dict = {}

class config:
    def __init__(self, *args, **kwargs):
        pass

    @staticmethod
    def get(key, default=None):
        return config_dict.get(key, default)

class this:
    """dbt.this() or dbt.this.identifier"""
    database = 'awsdatacatalog'
    schema = 'analytics_dev'
    identifier = 'my_first_dbt_model'
    def __repr__(self):
        return 'analytics_dev.my_first_dbt_model'

class dbtObj:
    def __init__(self, load_df_function) -> None:
        self.source = lambda *args: source(*args, dbt_load_df_function=load_df_function)
        self.ref = lambda *args: ref(*args, dbt_load_df_function=load_df_function)
        self.config = config
        self.this = this()
        self.is_incremental = False

# COMMAND ----------

def materialize(spark_session, df, target_relation):
    import pandas
    try:
        if isinstance(df, pandas.core.frame.DataFrame):
            df = spark_session.createDataFrame(df)
        df.write \
        .format("parquet") \
        .option("path", "s3://dbt-athena/analytics_dev/my_first_dbt_model/bd1ebe1c-e14f-4f6a-839c-d265353e15ce") \
        .mode("overwrite") \
        .saveAsTable(
            name="analytics_dev.my_first_dbt_model",
        )
        return "OK"
    except Exception:
        raise

dbt = dbtObj(spark.table)
df = model(dbt, spark)
materialize(spark, df, dbt.this)

23:24:49.277160 [debug] [Thread-2 (]: Athena adapter: Submitted calculation execution id 20c3a4b7-ffb7-dbed-4ffa-20f9fd060d07
23:24:49.792450 [debug] [Thread-1 (]: SQL status: OK -1 in 1 seconds
23:24:49.799724 [debug] [Thread-1 (]: Using athena connection "model.dbt_athena_spark.model"
23:24:49.800766 [debug] [Thread-1 (]: On model.dbt_athena_spark.model: alter table `analytics_dev`.`model` set tblproperties ('classification' = 'parquet')
23:24:51.256504 [debug] [Thread-1 (]: SQL status: OK -1 in 1 seconds
23:24:51.270813 [debug] [Thread-1 (]: Timing info for model.dbt_athena_spark.model (execute): 2023-04-03 23:24:43.522016 => 2023-04-03 23:24:51.270699
23:24:51.271712 [debug] [Thread-1 (]: On model.dbt_athena_spark.model: Close
23:24:51.273239 [debug] [Thread-1 (]: Sending event: {'category': 'dbt', 'action': 'run_model', 'label': '3d6bdb09-bba6-4b8c-9bdc-ff76eb1afb75', 'context': [<snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x7efeca585ab0>]}
23:24:51.274397 [info ] [Thread-1 (]: 1 of 3 OK created sql table model analytics_dev.model .......................... [OK -1 in 7.80s]
23:24:51.277656 [debug] [Thread-1 (]: Finished running node model.dbt_athena_spark.model
23:24:51.279058 [debug] [Thread-4 (]: Began running node model.dbt_athena_spark.my_incremental_dbt_model
23:24:51.280067 [info ] [Thread-4 (]: 3 of 3 START python incremental model analytics_dev.my_incremental_dbt_model ... [RUN]
23:24:51.281391 [debug] [Thread-4 (]: Acquiring new athena connection 'model.dbt_athena_spark.my_incremental_dbt_model'
23:24:51.282138 [debug] [Thread-4 (]: Began compiling node model.dbt_athena_spark.my_incremental_dbt_model
23:24:51.293378 [debug] [Thread-4 (]: Writing injected SQL for node "model.dbt_athena_spark.my_incremental_dbt_model"
23:24:51.294473 [debug] [Thread-4 (]: Timing info for model.dbt_athena_spark.my_incremental_dbt_model (compile): 2023-04-03 23:24:51.282741 => 2023-04-03 23:24:51.294358
23:24:51.295253 [debug] [Thread-4 (]: Began executing node model.dbt_athena_spark.my_incremental_dbt_model
23:24:51.365718 [debug] [Thread-4 (]: temporary relation isanalytics_devmy_incremental_dbt_model__dbt_tmp
23:24:51.367263 [debug] [Thread-4 (]: Opening a new connection, currently in state init
23:24:51.897433 [debug] [Thread-4 (]: Athena adapter: Error calling Glue get_table: An error occurred (EntityNotFoundException) when calling the GetTable operation: Table my_incremental_dbt_model__dbt_tmp not found.
23:24:52.640014 [debug] [Thread-4 (]: Athena adapter: S3 path does not exist
23:24:52.641257 [debug] [Thread-4 (]: On model.dbt_athena_spark.my_incremental_dbt_model:

  import pandas as pd

def model(dbt, session):
    dbt.config(materialized="incremental")
    df = dbt.ref("model")

    if dbt.is_incremental:
        max_from_this = f"select max(run_date) from {dbt.this}"
        df = df.filter(df.run_date >= session.sql(max_from_this).collect()[0][0])

    return df

# This part is user provided model code
# you will need to copy the next section to run the code
# COMMAND ----------
# this part is dbt logic for get ref work, do not modify

def ref(*args,dbt_load_df_function):
    refs = {"model": "analytics_dev.model"}
    key = ".".join(args)
    return dbt_load_df_function(refs[key])

def source(*args, dbt_load_df_function):
    sources = {}
    key = ".".join(args)
    return dbt_load_df_function(sources[key])

config_dict = {}

class config:
    def __init__(self, *args, **kwargs):
        pass

    @staticmethod
    def get(key, default=None):
        return config_dict.get(key, default)

class this:
    """dbt.this() or dbt.this.identifier"""
    database = 'awsdatacatalog'
    schema = 'analytics_dev'
    identifier = 'my_incremental_dbt_model'
    def __repr__(self):
        return 'analytics_dev.my_incremental_dbt_model'

class dbtObj:
    def __init__(self, load_df_function) -> None:
        self.source = lambda *args: source(*args, dbt_load_df_function=load_df_function)
        self.ref = lambda *args: ref(*args, dbt_load_df_function=load_df_function)
        self.config = config
        self.this = this()
        self.is_incremental = True

# COMMAND ----------

def materialize(spark_session, df, target_relation):
    import pandas
    try:
        if isinstance(df, pandas.core.frame.DataFrame):
            df = spark_session.createDataFrame(df)
        df.write \
        .format("parquet") \
        .option("path", "s3://dbt-athena/analytics_dev/my_incremental_dbt_model__dbt_tmp/024a5a18-9176-477e-9ef6-71a912166b6c") \
        .mode("overwrite") \
        .saveAsTable(
            name="analytics_dev.my_incremental_dbt_model__dbt_tmp",
        )
        return "OK"
    except Exception:
        raise

dbt = dbtObj(spark.table)
df = model(dbt, spark)
materialize(spark, df, dbt.this)

23:24:53.376039 [debug] [Thread-4 (]: Athena adapter: Submitted calculation execution id 00c3a4b8-07b8-b91c-7333-4444fd3b2875
23:25:03.888763 [debug] [Thread-2 (]: Athena adapter: Received execution status COMPLETED
23:25:03.976429 [debug] [Thread-2 (]: Execution status: OK in 15.62 seconds
23:25:03.978580 [debug] [Thread-2 (]: Timing info for model.dbt_athena_spark.my_first_dbt_model (execute): 2023-04-03 23:24:43.542495 => 2023-04-03 23:25:03.978517
23:25:03.979020 [debug] [Thread-2 (]: On model.dbt_athena_spark.my_first_dbt_model: Close
23:25:03.979870 [debug] [Thread-2 (]: Sending event: {'category': 'dbt', 'action': 'run_model', 'label': '3d6bdb09-bba6-4b8c-9bdc-ff76eb1afb75', 'context': [<snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x7efec9415420>]}
23:25:03.980409 [info ] [Thread-2 (]: 2 of 3 OK created python table model analytics_dev.my_first_dbt_model .......... [OK in 20.51s]
23:25:03.980877 [debug] [Thread-2 (]: Finished running node model.dbt_athena_spark.my_first_dbt_model
23:25:07.865205 [debug] [Thread-4 (]: Athena adapter: Received execution status COMPLETED
23:25:08.007879 [debug] [Thread-4 (]: Execution status: OK in 15.37 seconds
23:25:08.486506 [debug] [Thread-4 (]: Athena adapter: Columns in relation my_incremental_dbt_model: [{'Name': 'column_1', 'Type': 'int', 'Comment': ''}, {'Name': 'column_2', 'Type': 'int', 'Comment': ''}, {'Name': 'run_date', 'Type': 'varchar(10)', 'Comment': ''}]
23:25:08.488714 [debug] [Thread-4 (]:

    insert into "analytics_dev"."my_incremental_dbt_model" ("column_1", "column_2", "run_date")
    (
       select "column_1", "column_2", "run_date"
       from "analytics_dev"."my_incremental_dbt_model__dbt_tmp"
    );
23:25:08.492212 [debug] [Thread-4 (]: Writing runtime python for node "model.dbt_athena_spark.my_incremental_dbt_model"
23:25:08.494996 [debug] [Thread-4 (]: On model.dbt_athena_spark.my_incremental_dbt_model:

23:25:09.347206 [debug] [Thread-4 (]: Athena adapter: Submitted calculation execution id d8c3a4b8-26e7-e6a5-cbcc-e64157ed6fbf
23:25:09.476405 [debug] [Thread-4 (]: Athena adapter: Received execution status COMPLETED
23:25:09.560324 [debug] [Thread-4 (]: Execution status: OK in 1.06 seconds
23:25:10.052199 [debug] [Thread-4 (]: Athena adapter: table_name : my_incremental_dbt_model__dbt_tmp
23:25:10.052961 [debug] [Thread-4 (]: Athena adapter: table type : table
23:25:10.551817 [debug] [Thread-4 (]: Athena adapter: analytics_dev.my_incremental_dbt_model__dbt_tmp is stored in s3://dbt-athena/analytics_dev/my_incremental_dbt_model__dbt_tmp/024a5a18-9176-477e-9ef6-71a912166b6c
23:25:10.918950 [debug] [Thread-4 (]: Athena adapter: Deleting table data: path='s3://dbt-athena/analytics_dev/my_incremental_dbt_model__dbt_tmp/024a5a18-9176-477e-9ef6-71a912166b6c', bucket='dbt-athena', prefix='analytics_dev/my_incremental_dbt_model__dbt_tmp/024a5a18-9176-477e-9ef6-71a912166b6c/'
23:25:11.408505 [debug] [Thread-4 (]: Using athena connection "model.dbt_athena_spark.my_incremental_dbt_model"
23:25:11.410070 [debug] [Thread-4 (]: On model.dbt_athena_spark.my_incremental_dbt_model: drop table if exists `analytics_dev`.`my_incremental_dbt_model__dbt_tmp`
23:25:13.086708 [debug] [Thread-4 (]: SQL status: OK -1 in 2 seconds
23:25:13.091217 [debug] [Thread-4 (]: Timing info for model.dbt_athena_spark.my_incremental_dbt_model (execute): 2023-04-03 23:24:51.295881 => 2023-04-03 23:25:13.091109
23:25:13.092127 [debug] [Thread-4 (]: On model.dbt_athena_spark.my_incremental_dbt_model: Close
23:25:13.093652 [debug] [Thread-4 (]: Sending event: {'category': 'dbt', 'action': 'run_model', 'label': '3d6bdb09-bba6-4b8c-9bdc-ff76eb1afb75', 'context': [<snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x7efea76d69e0>]}
23:25:13.094774 [info ] [Thread-4 (]: 3 of 3 OK created python incremental model analytics_dev.my_incremental_dbt_model  [OK in 21.81s]
23:25:13.095632 [debug] [Thread-4 (]: Finished running node model.dbt_athena_spark.my_incremental_dbt_model
23:25:13.098636 [debug] [MainThread]: Acquiring new athena connection 'master'
23:25:13.100103 [debug] [MainThread]: Connection 'master' was properly closed.
23:25:13.100840 [debug] [MainThread]: Connection 'model.dbt_athena_spark.model' was properly closed.
23:25:13.101504 [debug] [MainThread]: Connection 'model.dbt_athena_spark.my_first_dbt_model' was properly closed.
23:25:13.102163 [debug] [MainThread]: Connection 'model.dbt_athena_spark.my_incremental_dbt_model' was properly closed.
23:25:13.104309 [info ] [MainThread]:
23:25:13.105121 [info ] [MainThread]: Finished running 2 table models, 1 incremental model in 0 hours 0 minutes and 30.96 seconds (30.96s).
23:25:13.106144 [debug] [MainThread]: Command end result
23:25:13.120593 [info ] [MainThread]:
23:25:13.121505 [info ] [MainThread]: Completed successfully
23:25:13.122211 [info ] [MainThread]:
23:25:13.122947 [info ] [MainThread]: Done. PASS=3 WARN=0 ERROR=0 SKIP=0 TOTAL=3
23:25:13.123752 [debug] [MainThread]: Sending event: {'category': 'dbt', 'action': 'invocation', 'label': 'end', 'context': [<snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x7efec9266290>, <snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x7efea76a3df0>, <snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x7efea76a3a60>]}
23:25:13.124622 [debug] [MainThread]: Flushing usage events

Tests added

Check if session id is obtained

TODO:

Check if calculation is submitted
Check if execution is polled

Checklist

[ ] You followed contributing section
[ ] You added unit testing when necessary
[ ] You added functional testing when necessary

Overall LGTM, we should add some documentation on:

the requirements
how it works

@Jrmyy I have updated the README with instructions and also provided docstrings to explain what happens behind the scenes. Let me know if I need to add more details.

@Avinash-1394 have a lock at monkeypatch to mock specific calls from a function - https://www.patricksoftwareblog.com/monkeypatching-with-pytest/
Regarding the issue with cached_property I left you a suggestion, it's should work, but I didn't test it.

@nicor88 I have added one test but mock_athena does not have support for these client calls since they were added very recently. Its making it a little difficult to test these. Reference
I am going to try and contribute to moto for these. But can we merge this and do that in a future MR? Or should we try to add more tests before we can merge this?

@Avinash-1394 what you did here is an amazing feature, regarding mocking the requests could be done with mock patch, but let's consider to open an issue in moto and add a Todo in the tests.
I want to take some more time to review the changes proposed here, also @Jrmyy @mattiamatrix could you have a look too? Ideally we include this in the next release :)


Extract getContributors to a support class

closes #240
This PR extract the getContributors method from the AboutController to a support class so it can be mocked and used elsewhere if needed.

Nice!


The --maximum_iteration flag doesn't actually do anything

I was trying to limit the time waiting for the generation of the test cases, and for that I used the flag. Nonetheless, the flag didn't limit the number of iterations of the program.

I just tested this on git tag 0.27.0 and still seems to work for me, i.e.,
I get the following (truncated) output
[17:14:34] [INFO](pynguin.generator:_run:507): Start generating test cases
[17:14:35] [INFO](pynguin.generation.searchobserver:before_first_search_iteration:66): Initial Population, Coverage: 0.750000
[17:14:35] [INFO](pynguin.generation.searchobserver:after_search_iteration:70): Iteration:       1, Coverage: 0.750000
[17:14:35] [INFO](pynguin.generator:_run:512): Stopping condition reached
[17:14:35] [INFO](pynguin.generator:_run:514): Used iterations: 1/1
[17:14:35] [INFO](pynguin.generator:_run:515): Stop generating test cases

when I specify --maximum_iterations 1 .
Can you verify that you are using the most recent version of Pynguin?

Thanks for the quick reply. I think the main problem in question is that i installed it in Ubuntu and the version that pip gets from it is the 0.17 version. I also noticed that in this version there is no such thing as --maximum-search-time, and, instead, there's the flag --stopping_condition MAX_ITERATIONS which fixes it.
TLDR: The problem was that when installing from pip in Ubuntu i can only get version 0.17 instead of 0.27.

I don't know about Ubuntu and its software versions but I suspect that your Ubuntu provides Python 3.8 or 3.9. Pynguin 0.17 was the last version supporting these Python versions, we dropped the support in favour of only supporting Python 3.10 with Pynguin 0.18.0. If your Ubuntu only provides Python < 3.10, pip won't be able to install a more recent version of Pynguin.
You might want to consider using Pynguin in a Docker container, which would allow you to use more recent versions. Alternatively, may I suggest to consider using pyenv to install various versions of Python easily?

Yes, thank you! My Ubuntu is using python 3.8. Fixed the issue, thank you very much!


Guide translations

Refactor topics of data.json to have object of keys xh, zu and en with topic array in those keys
Create custom hooks to detect language changes return current language used
Make refactors through code base that use guides from  data.json(aka topics)

@musangowope this is good to merge.


chore(trading): upgrade pennant to v1.9.0

Related issues 🔗
Closes #4234
Description ℹ
Enables the trading app to show correctly formatted volume numbers in candlestick charts.
Demo 📺

Hey @johnwalley this branch failed tests due to something I broke, which is now fixed (see #4232). A quick rebase should fix it.


Looking for new maintainers!

If you currently use Fs2 Rabbit, please consider becoming a maintainer.
I no longer have neither time nor interest to work on this since it's been about 3 years since last time I used RabbitMQ in production.
The project started as a private project in early 2017 and a few months later I made it open source. I'm overwhelmed with how fast it grew up and didn't expect many users to start using a simple client that started from my simple necessities.
This is OSS so becoming a maintainer means dedicating as much time as you want to it, there's no work schedule.
If you're interested, get in touch via Gitter or reply here. @Daenyth is one of the active users that has been helping lead the project for a while but his time is also limited hence this message to all of you.

Hey @gvolpe, I use fs2-rabbit a lot and would love to help out as a maintainer

Seconding @agustafson, would also like to offer to help out!

Awesome guys! Added you both as maintainers :)

Hi @jbwheatley @agustafson @gvolpe I use fs2-rabbit at work and i can help as a maintainer

I have not used rabbitMQ for work for over a year, so don't feel in a place to effectively maintain this repo anymore. If the keys to the city could be passed to someone else, that would be appreciated @gvolpe @Daenyth

@gvolpe I don't see many activities here, I can also help if needed. We use this library in production and I'd like to contribute and help

@matejcerny tagging also you here

I can add you as a maintainer, but we need to fix this https://github.com/profunktor/fs2-rabbit/issues/931. Without it, we are unable to publish any change

I have not used rabbitMQ for work for over a year, so don't feel in a place to effectively maintain this repo anymore. If the keys to the city could be passed to someone else, that would be appreciated @gvolpe @Daenyth

I'm going to remove myself as an admin


pip installation issue

When I run pip install -r requirements.txt in kali linux I get the following.

I also got the same error for textwrap

@hlldz Now there is nothing in main menu.

@hlldz Working fine now.Thanks


Use divided tax limitations 15%

Closes #3

Added support for export detail info for dividends in separate Countries and Brokers


fix(eth-watch): Do not track for stm, only for diamond proxy

What ❔
We plan to generally have all upgrades executed via a direct call to diamond proxy.
While in principle it is possible that an upgrade comes via a direct call through STM, this is a very rare circumstance and for now out of scope
Why ❔

Checklist

[ ] PR title corresponds to the body of PR (we generate changelog entries from PRs).
[ ] Tests for the changes have been added / updated.
[ ] Documentation comments have been added / updated.
[ ] Code has been formatted via zk fmt and zk lint.
[ ] Spellcheck has been run via zk spellcheck.

@perekopskiy the first one is still correct as only ops from the governance are tracked


Out of date page layout

Azure documentation issue guidance
Thanks for opening an issue in the Azure technical documentation repository.
We use GitHub issues as the primary channel for customer and community feedback about the Azure documentation.
Creating an issue
We prefer that you create documentation feedback issues using the Feedback link on the published article - the feedback control on the doc page creates an issue that contains all the article details so you can focus on the feedback part.
You can also create a feedback issue here in the repo. If you do this, please make sure your issue lists:

[ ] The relevant Azure service or technology.
[ ] A link to the published documentation article that you have feedback about.
[ ] Clear, specific feedback that the author can act on.

Pull requests and article contributions
If you know the change that is needed in an article, we encourage you to submit the changes directly using a pull request. If the change is large, or if you want to contribute an entire article, follow these guidelines:

[ ] Don't surprise us with a big pull request or a pull request with a new article! Submit an issue that describes the details of the proposed large change or new article.
[ ] Include the service or technology area.

We'll route the issue to the appropriate content team for review and discussion.
Tech support and product feedback
If you would like to contact Microsoft about other things, such as product feedback or tech support, please review these guidelines:

If you need technical support using Azure, the paid and free support options are described here: https://azure.microsoft.com/support/options/.

Each article in the Azure technical documentation contains a product feedback button - it's best to submit product feedback directly from a relevant article. Otherwise, you can submit product feedback for most Azure products in the following product feedback forum: https://feedback.azure.com/d365community/forum/79b1327d-d925-ec11-b6e6-000d3a4f06a4.

@RobDatesand
Thanks for posting this. Based on that shared information we are not able to understand your query/feedback completely. Since this channel is for driving improvements towards MS Docs, could you provide the URL of the specific documentation that your feedback is applicable to? That way, we are able to make sure your feedback is addressed by the right team.

<EMAIL_ADDRESS>I tackled this by deleting the function app and creating a new one. This part of the configuration then appeared. You're right this appears to be a bug with the azure GUI rather than the documentation.


Installation Question

I'd really like to use this polyfill, but the installation instructions shock me and I hope I've misunderstood them. Do I really need (on windows 7) to:

Install (outdated) 2.7.x Python
Install Visual Studio C++ Express
Install node-gyp
Make sure Python is available in Path
Install Windows 7 SDK for 64-bit-support
Install (outdated) GTK 2
build node-gyp
npm install postcss-conic-gradient --save-dev

Isn't there a simpler way?

Isn't there a simpler way?

Yes, you can use the prefixfree polyfill and not install anything :)

The next release will not require this. It just took a few years for amazing people to rewrite canvas in JavaScript and WASM.


Operations from conditions execute after the operation's dependencies

This PR implements the changes as discussed in #147. Essentially each Operation instance now has an optional "waiter" NSOperation property internally.
We also now have the concept of direct dependency (or maybe regular dependency), and an indirect dependency. A direct dependency is one added via addDependency whereas an indirect dependency is the dependency of an operation condition.
When addDependency is invoked, the argument operation is added as a dependency to an internal NSOperation instance which is created on the fly (and added as a dependency of the receiver using super.addDependency). Essentially this creates a dummy operation which acts as a surrogate for the receiver. It waits for all the direct dependencies.
When removeDependency is invoked, this structure is broken down.
When accessing dependencies the array of dependencies has the waiter operation removed, but we add all of its dependencies to the array.
When an Operation is added to an OperationQueue, as before we extract the dependency operations from the conditions. For each of these dependencies we use an internal addConditionDependency function on the operation. If the receiver has a waiter set (i.e. it previously had direct dependencies added), we add the waiter as a dependency of the condition before adding the condition as a dependency of the receiver (using super.addDependency).
The diagram below represents the execution of operations on the queue for the green Operation instance.

Current coverage is 67.57%

Merging #157 into development will decrease coverage by -0.30% as of 53e784b

@@            development    #157   diff @@
===========================================
  Files                51      51
  Stmts              2792    2791     -1
  Branches              0       0
  Methods
===========================================
- Hit                1895    1886     -9
  Partial               0       0
- Missed              897     905     +8

Review entire Coverage Diff as of 53e784b

Powered by Codecov. Updated on successful CI builds.


Further fix for BZ 66508

https://bz.apache.org/bugzilla/show_bug.cgi?id=66508
Avoid deadlock for close messages when
WsRemoteEndpointImplServer.endMessage() for a previous message is processed on a container thread
The fix for this involves the thread currently processing a socket effectively yielding the socket lock and processor to another thread. I think that is worthy of closer review before merge.
There are a couple of potential alternatives.
Plan B would be to make handing of the close message async. That is potentially better but will be more complex to implement and may uncover other complexities.
Plan C would be to require useAsyncIO="true" to enable WebSocket.

Adding workarounds for this should be good (I suppose you have the means to test it on hand ;) ). I like the original goal of websockets to be built on the Servlet API, but there are some known difficulties for this.
Of course, using async IO should always be recommended now that it works, since it is going to be more efficient in the "I'm doing real IO" scenario (more lightweight and straightforward).

I have a test in that I can manually trigger the issue with a debugger at the moment and can step through the code with the fix to confirm it works. My desire is to create a unit test the recreates the issue. I have an idea for that but haven't implemented it yet.

Updated and applied directly.


Update parsing for parseStudents

Closes #185
Closes #195
Closes #202

The various error checking and messages are added:

``: Student argument cannot be empty!
,, 0, e012345a: Student argument is invalid!

I'm unable to specify that 0 is out of bounds as I used StringUtil#isNonZeroUnsignedInteger()

1,a: Index is not a non-zero unsigned integer.
123: Student index is out of bounds!
e0123456,e012345a: Student argument E012345A is invalid!
e0123450, e0123456,e0123450 (2nd student not found): Student does not exist in the list!

I'm unable to specify which student does not exist

Codecov Report

Merging #213 (a82afce) into master (a530d40) will decrease coverage by 0.05%.
The diff coverage is 0.00%.

@@             Coverage Diff              @@
##             master     #213      +/-   ##
============================================
- Coverage     27.65%   27.59%   -0.06%
  Complexity      409      409
============================================
  Files           148      148
  Lines          3327     3334       +7
  Branches        394      397       +3
============================================
  Hits            920      920
- Misses         2342     2349       +7
  Partials         65       65

Impacted Files
Coverage Δ

...in/java/seedu/address/logic/parser/ParserUtil.java
17.56% <0.00%> (-0.88%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a530d40...a82afce. Read the comment docs.


Release v1.20.0

This is a draft of changelog for the v1.20.0 release.
Here you can find all related translations changes: v1.19.0...master.

⚠ Do not merge this pull request manually ⚠
You should approve this pull request instead. After approving, I will automatically update it (release date needs to be adjusted), merge, and tag a new release. You can read more about the process on forum.

Show release notes preview
v1.20.0

新增翻譯:

flarum/extension-manager

更新翻譯:

fof/recaptcha

全部變化：v1.19.0...v1.20.0

Success! Now you can announce new release on forum:
## 版本 [`v1.20.0`](https://github.com/flarum-lang/chinese-traditional/releases/tag/v1.20.0)

**新增翻譯**:

* [`flarum/extension-manager`](https://github.com/flarum/extension-manager)

**更新翻譯**:

* [`fof/recaptcha`](https://github.com/FriendsOfFlarum/recaptcha)

全部變化：[v1.19.0...v1.20.0](https://github.com/flarum-lang/chinese-traditional/compare/v1.19.0...v1.20.0)

更新命令：

```console
composer update flarum-lang/chinese-traditional
php flarum cache:clear
```


Could not load file or assembly 'Vanara.Windows.Shell'... manifest definition does not match the assembly reference

I've been getting quite a few crashes in telemetry with this content:
System.IO.FileLoadException: Could not load file or assembly 'Vanara.Windows.Shell, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=c37e4080322237fa' or one of its dependencies. The located assembly's manifest definition does not match the assembly reference. (Exception from HRESULT: 0x80131040)
    at   (IconsManager.cs:109)

pointing to a completely irrelevant part of the code:
BitmapImage bi = new BitmapImage();
bi.BeginInit(); //<- ----------here
bi.UriSource = new Uri("pack://application:,,,/MyApp;component/Images/icon.png", UriKind.Absolute);

and that static class doesn't even have Vanara.* in the using
I had this with <IP_ADDRESS> also.
.NET4.8
Added through NuGet
Do you have any suggestions on how to determine the cause of this?

This is curious. If you can, will you try replacing Vanara.Windows.Shell with Vanara.Windows.Shell.Common 3.4.0 from the NuGet repository at https://ci.appveyor.com/nuget/vanara-prerelease? We've dramatically reduced dependencies and I'm interested to know if that helps your issue.

After getting more detailed reports it looks like this is happening for users who updated from the version that used Vanara.Windows.Shell version <IP_ADDRESS>, but the updater should be replacing all the files including that dll, so I am not sure if that is relevant. But I will pack a version with Vanara.Windows.Shell.Common and get back with findings

Version 3.4.0 was just released. Let me know if your problem persists.

I have switched to Vanara.Windows.Shell.Common and I didn't get any System.IO.FileLoadException anymore. I like that it has less dependencies.
I did get a a different error but I have to check first if those are just users deleting dependencies

@milos12345 If your issue is resolved, please close it. If not, let me know what you're finding.


A sub-exposure that failed the cleaning stage appears to be being processed further

0783270101PNU002 fails espfilt (see issue #133 for why), but there is a PNU002_clean.fits file in the archive output for that observation, obviously shouldn't be as the GTI can't be applied because it can't exist!

With the work I've done on issue #135 (with a couple more processing methods to add the new past process success checking method to) the situation described in this issue should no longer occur.


Migrate console-frontend to trunk

Webpack and the wasm-pack plugin for webpack are a constant source of troubles. On the other side, trunk has evolved pretty nicely, and some initial tests look promising.
We should try to replace the webpack build with trunk. This requires splitting off the interactive API console, which is based on SwaggerUI, using React, and webpack too.
However, if we want to add additional APIs in the future, which are not part of our own API specification, we might need a different approach anyway. So let's create a dedicated API UI. And link that into the console, as we do it now. But, separate the internally.
Fixes #111

Blocked by: https://github.com/thedodd/trunk/issues/286


cmake: install mavlink headers in correct dir

Taken from https://github.com/mavlink/MAVSDK/commit/0c8e2367952b3233669739d439a13f2ccb568470, as it feels useful to merge it ASAP.

I work with a deb package build locally on my side it's all good, but if others try to work with the deb provided on github it won't work

@julianoes Would it be possible to do a patch release ? Or do you have a plan to integrate this correction in v1.5.0 and when could we except to be released ?

Thanks for the note @Katawann. Here comes a patch for v1.4: #1911.

I have the feeling this probably is not required for v1.4. It seems to be broken in v1.4.7. I'll revert it.
The changes in main will eventually be released as v2.

Arf too bad. I'll wait for new release to test the installation then. Thank you for your help anyway !

But it works in main, doesn't it?

I need to test again because I need to build the package to include it in my docker. I come back to you when I found some time to test it

Got it! Let us know if you find a patch, would be helpful :blush:

I just build debian package with main branch but it still fail with the same error:
/usr/include/mavsdk/plugins/mavlink_passthrough/mavlink_include.h:3:10: fatal error: mavlink/v2.0/common/mavlink.h: No such file or directory
#include "mavlink/v2.0/common/mavlink.h"

Now let's found a fix which does not break installation for others...

Ok, so takeaway: main is broken, release should (again be fine) as it was.


Adding model and partitioner for remote deployments

Created the model for splitting a flow over multiple deployments.

Can you rebase and/or cherry-pick?  This PR contains a commit that's already in the master branch, from Simon


Refactor DGW

DO NOT MERGE

Having seek for every potential usage of this lib, I didn't found any, DAPI might be the only one to use it firstly, so I don't feel that a major bump for a renamed method is a problem any more :)

Thanks.   Actually do not merge this yet.   I consider to enable the lib to accept headers with difficulty bits provided and not target as it is currently.  This will make it directly compatible with the stored BlockHeaders stored on a client's device.

@KamuelaFranco changes updated as per suggestions.  If you agree with current state please merge :)


[TODO] feature: build dataset from Anki's sqlite database

Python implementation:
https://github.com/open-spaced-repetition/fsrs-optimizer/blob/95694b787bb71ac9883db1201af09e334ee5ee0b/src/fsrs_optimizer/fsrs_optimizer.py#L319-L449

DataFrame library for Rust: https://docs.rs/polars/latest/polars/

I wonder if this could perhaps be accomplished using standard Rust iterators/structures and SQL? Eg fetching the data from the revlog table in the desired order, and collecting it into a Vec<(CardId, Vec)> or similar. Upsides would be avoiding the addition of another non-trivial dependency, and the use of typed structs would make the code somewhat more maintainable than looking up columns by string keys. I'm not sure how easily all of those dataframe calls could be translated though, so don't know if it's practical or not.

OK, I will try. Maybe two main loops will implement it. I use pandas in Python just for efficiency and convenience.

I have tried to query revlog from Anki db in 5a051e9834abfa8309a50e2aadb751c46b0daa3c.
RevlogEntry {
    id:<PHONE_NUMBER>777,
    cid:<PHONE_NUMBER>309,
    usn: 1282,
    button_chosen: 4,
    interval: 3,
    last_interval: -60,
    ease_factor: 2500,
    taken_millis: 3918,
    review_kind: 0,
}

The next is to rewrite these codes:
df = pd.DataFrame(revlog)
df.columns = ['id', 'cid', 'usn', 'button_chosen', 'interval', 'last_interval', 'ease_factor', 'taken_millis', 'review_kind']

df = df[(df['cid'] <= time.time() * 1000) &
        (df['id'] <= time.time() * 1000)].copy() # remove revlog with incorrect timestamp

df_set_due_date = df[(df['review_kind'] == 4) & (df['interval'] > 0)]
df.drop(df_set_due_date.index, inplace=True) # remove revlog generated by `set due date`

df.sort_values(by=['cid', 'id'], inplace=True, ignore_index=True) # sort revlog by card_id, review_time

df['is_learn_start'] = (df['review_kind'] == 0) & (df['review_kind'].shift() != 0)  # find out the first review for each card
df['sequence_group'] = df['is_learn_start'].cumsum() # if the user never used `forget` in a card, the revlogs from the same card should has the same `sequence_group`
last_learn_start = df[df['is_learn_start']].groupby('cid')['sequence_group'].last() # get the `sequence_group` for each card's `last_learn_start`
df['last_learn_start'] = df['cid'].map(last_learn_start).fillna(0).astype(int)
df['mask'] = df['sequence_group'] >= df['last_learn_start'] # remove the revlogs which happen before the `last_learn_start`
df = df[df['mask'] == True].copy()

df = df[(df['review_kind'] != 4)].copy() # remove revlog generated by  `reschedule_cards_as_new`
df = df[(df['review_kind'] != 3) | (df['ease_factor'] != 0)].copy() # remove revlog in filtered decks with rescheduling disabled
df['review_kind'] = df['review_kind'] + 1
df.loc[df['is_learn_start'], 'review_kind'] = New # New = 0
df.drop(columns=['is_learn_start', 'sequence_group', 'last_learn_start', 'mask', 'usn', 'interval', 'last_interval', 'ease_factor'], inplace=True)

Done in 19dc456680d950191f1c929c89eeed46a00e7314.
@dae, I think the next step is to determine the API?
You can test the training here:
https://github.com/open-spaced-repetition/fsrs-optimizer-burn/blob/19dc456680d950191f1c929c89eeed46a00e7314/src/training.rs#L132-L146
The file collection.anki21 should be placed in the directory tests/data/.

You work fast! :-)
I gave this a quick test on a collection with about 200k revlog entries, and it completed in around 90s. I tried a different collection with about 1M entries, and performance did not scale linearly - at the 8 minute mark it was still early on epoch 2. How have you found performance to compare to training with CPU/GPU in PyTorch? If you weren't already aware, using 'cargo test --release' will make sure it's compiled in release mode which should be faster.
In terms of integration into Anki, I imagine we could have a collection method inside the anki crate that dumps the revlog into a vec of FSRSItems (1). We could then call a method exported by this crate that takes a Vec of those items, and it would process it and return the results. We could call it on a background thread, so users could continue to use Anki for most of the training. Users usually won't have access to a console window, so ideally we'd find some way to poll for the current progress/completion ratio, so the UI can show it. If temporary files need to be written out, it would be nice if the desired folder could be passed into the API call. Any config options that might make sense for the user to adjust should also be passed in to the call, so the frontend can expose a UI to adjust them in the future.
How are you feeling about the state of this at this point? Does it look like using burn might be a viable alternative to the current pytorch approach, at least if we can solve the clipping problem in #4?
(1) Re FSRSItem, is the length of t_history and r_history always the same? If so, perhaps it would make things clearer to store it as a reviews: Vec, where Review is a struct in the form { rating: i32; delta_time: i32; }? Then when it comes time to create a tensor, it could be done like
            Data::new(
                items.iter().map(|i| i.rating).collect(),
                Shape::new([items.len()]),
            )

Oh, and I checked the resulting size of a release build, and it's about 11-14MB (including sqlite). Much more viable than tch-rs!
I also played around with the wgpu backend, but it was much slower than ndarray on the Linux and M2 systems I tested with.

so ideally we'd find some way to poll for the current progress/completion ratio

If you can provide a synchronous/blocking code example that prints some text for each iteration/epoch, I can convert it so that it writes the progress to a shared data structure protected by a mutex, which we could pass in to the API call.

I gave this a quick test on a collection with about 200k revlog entries, and it completed in around 90s. I tried a different collection with about 1M entries, and performance did not scale linearly - at the 8 minute mark it was still early on epoch 2.

Your observation is consistent with my inference. The performance doesn't scale linearly. The reason is:\

For a card with 10 revlogs, the convertor will generate ~10 FSRSItems.
For these FSRSItems, the lengths of t_history/r_history are range from 1 to ~10.
In the training stage, the time complexity is linear to the seq_len.

So, for a card with n revlogs, the time complexity is 1+2+3+...+n = n(n+1)/2, i.e. O(n^2)
Consider two extreme cases: one with a collection with 'n' revlogs where each card has only one revlog, and another with just one card that includes all revlogs. The calculation complexity for these cases would be 'n' and 'n^2', respectively.

In terms of integration into Anki, I imagine we could have a collection method inside the anki crate that dumps the revlog into a vec of FSRSItems (1). We could then call a method exported by this crate that takes a Vec of those items, and it would process it and return the results.

I have implemented a method to convert revlogs to FSRSItems:
https://github.com/open-spaced-repetition/fsrs-optimizer-burn/blob/19dc456680d950191f1c929c89eeed46a00e7314/src/convertor.rs#L210-L221
But it needs to convert revlogs card by card. If the revlogs are incomplete for a card, the data will be invalid and removed.

How are you feeling about the state of this at this point? Does it look like using burn might be a viable alternative to the current pytorch approach, at least if we can solve the clipping problem in #4?

I'm pretty satisfied with burn, except for the over encapsulation of training process. I couldn't implement my own training loop. Fortunately, they plan to support it (https://github.com/burn-rs/burn/issues/662#issuecomment-1684280084).
If it is supported, continuous learning will be feasible. For old collections, the full training only needs to apply once. Then FSRS model could be trained by each review (or a batch of reviews) in the subsequent learning.

(1) Re FSRSItem, is the length of t_history and r_history always the same? If so, perhaps it would make things clearer to store it as a reviews: Vec<Review>, where Review is a struct in the form { rating: i32; delta_time: i32; };? Then when it comes time to create a tensor, it could be done like

Yeah. len(t_history) == len(r_history) is always true. But if we store it as ``reviews: Vec, then we will get a tensor with shape [seq_len, batch_size, 2]. Its dimension is three. I don't know how difficult it is to deal with a 3D tensor in the forward`. It's easy in PyTorch, but hard in Burn (partly due to my poor knowledge about Burn and Rust).
https://github.com/open-spaced-repetition/fsrs-optimizer-burn/blob/19dc456680d950191f1c929c89eeed46a00e7314/src/model.rs#L100-L119

I may be missing something, but couldn't this perhaps be handled in the mapping of FSRSItem to Tensors? Eg  in fn batch(&self, items: Vec<FSRSItem>) -> FSRSBatch<B>, the separate t_historys/r_historys could be created from the single reviews vector, like in https://github.com/open-spaced-repetition/fsrs-optimizer-burn/issues/3#issuecomment-1685159804?

Oh, I forget that. Thanks for reminder. Do you mean to rewrite FSRSItem as the following code?
pub struct FSRSItem {
    pub reviews: Vec<Review>,
    pub delta_t: f32,
    pub label: f32,
}

struct Review {
    pub rating: i32,
    pub delta_time: i32
}

Yep, exactly. You could call it FSRSReview if you found it clearer.
One question is how FSRS treats reviews in different SRS programs. Does it compare the different rating values, or does it just treat them as true/false? Does it assume the rating will be 1-4?

One question is how FSRS treats reviews in different SRS programs. Does it compare the different rating values, or does it just treat them as true/false? Does it assume the rating will be 1-4?

I have made a draft for the schema of review logs:
https://github.com/open-spaced-repetition/fsrs-optimizer#review-logs-schema
If other SRS programs also use FSRS, it is not a problem to treat different rating values. If they don't we can mapping their ratings values to 1-4. That's what I have done in the comparison between FSRS and SM-15:
https://github.com/open-spaced-repetition/fsrs-vs-sm15#fsrs-vs-sm-15

How about we mark this one as done and continue tracking things in #27?


Fixed small typo

thanks a lot @vpfautz !

Codecov Report

Merging #36 into master will not change coverage.
The diff coverage is n/a.

@@          Coverage Diff           @@
##           master     #36   +/-   ##
======================================
  Coverage    47.4%   47.4%
======================================
  Files          49      49
  Lines        1078    1078
======================================
  Hits          511     511
  Misses        567     567

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8de8e79...15c58e6. Read the comment docs.


Generalise rhyolite in preparation for using Vessel, modernise a bunch of things.

please update the changelog

Superseded by #74


move certmanager test helper into certmanager module

We need adaption patches to the service operators (sorry I did not noticed this before):

nova
ansibleee-operator
infra
keystone
placement
manila
ovn
neutron
ironic
heat
horizon
dataplane
glance
switft


Failed to load resource: the server responded with a status of 520

FYI:
[Error] Failed to load resource: the server responded with a status of 520 () (styles.min.css, line 0)
<EMAIL_ADDRESS>seems down.
Web server is returning an unknown error Error code 520
Visit cloudflare.com for more information.
2024-04-12 09:56:59 UTC
You
Browser
Working
Amsterdam
Cloudflare
Working
unpkg.com
Host
Error

The server appears to be operational again. If this occurs frequently, it would be prudent to include the file within the package rather than relying on an external source.


The render handle is not yet assigned.

I am getting an exception when trying to render my component.
Any directional help would be appreciated.
Here is my test:
[Fact(DisplayName = "Test to see the Lookup Renders Properly - Select Grid Item")]
        public void LookupRendersCorrectlySelectGridItem()
        {
            // Arrange
            var active = true;
            var id = 1;
            var description = "Item 1";
            var name = "Item1";
            var lookupEntities = new List<LookupViewModel>() { new LookupViewModel() { Active = active, Id = id, Description = description, Name = name } };
            var tipClient = new Mock<ITipClient>();
            tipClient.Setup(x => x.GetJsonAsync<List<LookupViewModel>>(It.IsAny<string>())).Returns(Task.FromResult(lookupEntities));

            Services.AddSingleton<ITipClient>(tipClient.Object);
            Services.AddMockJsRuntime();
            Services.AddTelerikBlazor();

            var root = new TelerikRootComponent();

            // Act
            var cut = RenderComponent<Lookup>(
                CascadingValue(root),
                CascadingValue("GridData", lookupEntities)
                );

            // Assert
            Assert.NotNull(cut);
        }

The same error happens to me when I try to add a simple menu item dynamically. I am using Radzen.Blazor v3.9.6.

I hope we can get a more stable version of Radzen components for Blazor soon:

@RafaelMeloTrakx you really need to provide a lot more details for me to help. Ideally a minimal project that has just enough code to reproduce your problem.

I am having this issue it has been very challenging.
I have included a minimal example to show what has been happening.

ProblemExample.zip

@Modricagon you are new'ing up the Modal component directly. That is not supported by Blazor and will result in the exception you are seeing. This is not really a bUnit thing, this is fundamental to how Blazor works.

@Modricagon you are new'ing up the Modal component directly. That is not supported by Blazor and will result in the exception you are seeing. This is not really a bUnit thing, this is fundamental to how Blazor works.

Thanks for this! This really helped me understand what I was doing wrong. Instead I rendered the Modal component first with the "var cut = ctx.RenderComponent();" and then passed the "cut.Instance".
Thanks so much issue resolved.

@Modricagon - I have this exact same issue - any chance you could expand on what you did to resolve it please?
i.e. How did you change:
[Test]
public void Test1()
{
    ctx.RenderComponent<Wizard>(parameters =>
    {
        parameters.Add(p => p.ModalInstance, new Modal());
    });
}


Small refactor in preparation for RHTAP multi-ci

We'll soon be adding "github" and "gitlab" to the existing "jenkins" RHTAP CI support. This is a small step towards doing that.
Ref: https://issues.redhat.com/browse/EC-990
Ref: https://issues.redhat.com/browse/EC-991

Split out from #1215 since I want to merge this one soon-ish.


8.2.0 doesn't compile

I really can't tell from the source code of the commit why but I can't compile 8.2.0, reverting back to 8.1.0 works.
Calling FlutterForegroundTaskPlugin.addTaskLifecycleListener() gives an Unresolved reference error. Yes, the import is there:
import com.pravera.flutter_foreground_task.*

and interestingly, FlutterForegroundTaskLifecycleListener resolves all right, from the same namespace. If I don't call the lifecycle static, the same reference in GeneratedPluginRegistrant will be the next unresolved reference.
Flutter is the latest, as always.

@deakjahn
version 8.2.0, nothing changed other than lowering the minSdkVersion.
Try using the method below:
Android Studio Menu > File > Invalidate Caches > Optional All check > Restart

and

run flutter command
> flutter clean
> flutter pub get

@Dev-hwang We're a bit past that, thanks. :-) It's not a cache issue because I can recreate it at will, both with the two versions from the usual Pub cache and from local copies. On the other hand, it could well be something specific to Android Studio that I work in, or a specific Gradle version, or whatever...
In addition to the actual minSdk change, you modified the source here and there to accommodate the 21-23 difference and also changed a couple of things in PluginUtils.kt (also modifying function signatures). So, I took parallel local copies of 8.1 and 8.2 and started to methodically restore file after file to see which one makes the error go away.
It wasn't a single file, I had to replace two: ForegroundService.kt and PluginUtils.kt for the first successful compilation, so I have to go further with the checks.

@deakjahn
Did you find the cause of the problem?
import androidx.core.content.ContextCompat

Is it possible that the above class will cause problem?

No, I'm constantly banging my head into the wall while I'm pulling out my hair at the same time. :-D
It also seems that building from inside the IDE causes problems while building from the command line works. Also, I have two copies of your source files that, for all intents and purposes, only differ in tabs vs. spaces (I also checked with WinMerge, a very nice diff program) and one works, the other doesn't. Something very fishy here...
Having said that, I never had any similar issues with Android Studio, neither will all my other apps, neither with your module prior this last change. Absolutely insane... :-)

@deakjahn
ok. I don't have a computer right now so I can't work. I'll revert the commit tomorrow and work again. :sob:

No, don't do that yet unless others also experience the same issue.

Right now, I'm struggling this this. We all saw that error many a time for sure:
* What went wrong:
Execution failed for task ':app:compileGoogleplayDebugKotlin'.
> A failure occurred while executing org.jetbrains.kotlin.compilerRunner.GradleCompilerRunnerWithWorkers$GradleKotlinCompilerWorkAction
   > Compilation error. See log for more details

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

I simply can't locate where that log is and reading the whole Internet did't provide an answer. Also, where to put --stacktrace or --debug? Some say Settings > Build > Gradle-Android Compiler > Options. Actually, the description there also says so.

@Dev-hwang Rest assured, the disease in contagious, I can no longer compile the 8.1.0 either, now. :-)
I also tried to ask at https://stackoverflow.com/questions/78849727/how-to-diagnose-gradle-issues-in-flutter, we'll see. Don't revert it for the whole wide world, it's either something in my setup (although I have no idea what it could be) or we need Lt Columbo to investigate it.

@Dev-hwang It kinda solved itself, I wouldn't dare to say that I did. See over at SO for details. 8.2.0 works as well.