[FLINK-26985][runtime] Don't discard shared state of restored checkpoints

What is the purpose of the change
As described in the ticket, in LEGACY restore mode,
shared state of incremental checkpoints can be discarded
regardless of whether they were created by this job or not.
The bug was introduced in FLINK-24611. Before, reference count was maintained
for each entry;
"initial" checkpoints did not decrement this count, preventing their shared state from being discarded.
This PR makes SharedStateRegistry to:

remember the max checkpiont ID encountered during recovery
associate each state entry with a checkpoint ID that created it
only discard the entry if its createdByCheckpointID > highestRetainCheckpointID``

(1) is called from:

CheckpointCoordinator.restoreSavepoint - to cover initial restore from a checkpoint
SharedStateFactory, when building checkpoint store - to cover the failover case

Only CheckpointCoordinator does not seem sufficient, because a new checkpoint
can be created, from which the job can recover automatically, without calling restoreSavepoint.
(see DefaultExecutionGraphFactory.createAndRestoreExecutionGraph)
Verifying this change
ResumeCheckpointManuallyITCase in LEGACY restore mode
Does this pull request potentially affect one of the following parts:

Dependencies (does it add or upgrade a dependency): no
The public API, i.e., is any changed class annotated with @Public(Evolving): no
The serializers: no
The runtime per-record code paths (performance sensitive): no
Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn/Mesos, ZooKeeper: no
The S3 file system connector: no

Documentation

Does this pull request introduce a new feature? no
If yes, how is the feature documented? no

Thanks for the review @Myasuka ,

I think current solution looks a bit friable as it involes many changes to different APIs.

I'm not sure what exactly do you mean. Indeed, there are changes to multiple interfaces, mostly because of the need to pass RestoreMode from CheckpointRecoveryFactory to SharedStateRegistryFactory.  However, I think this is unavoidable. Or do you see any alternative?

I am mostly fine with the changes. Besides the comments left above, my main concern is why introducing "RestoreMode DEFAULT = NO_CLAIM", let me explain my concerns in the following:

I am fine with the changes along the path "CheckpointCoordinator#restoreSavepoint", including all the interface changes, e.t.c.
I am not fine with RestoreMode DEFAULT = NO_CLAIM and passed in EmbeddedCompletedCheckpointStore, StandaloneCompletedCheckpointStore, CheckpointResourcesCleanupRunner, EmbeddedHaServicesWithLeadershipControl. There, it should follow the normal failover paths assuming all checkpoints belong to the same job, and non-referenced are subsumed as normal. I do not think NO_CLAIM follows this way.
So I would suggest we introduce a fourth internal value to indicate the normal JM failover behavior
@Myasuka mentioned a good point that item 2 itself is not enough: it is possible that before the first checkpoint after recovery is able to be made, JM crashes again, so item 2 may lead to losing information about whether the previous checkpoint is from a previous job or previous run.

We can also discuss offline.

Thanks for the review @curcur, regarding

I am not fine with RestoreMode DEFAULT = NO_CLAIM and passed in EmbeddedCompletedCheckpointStore, StandaloneCompletedCheckpointStore, CheckpointResourcesCleanupRunner, EmbeddedHaServicesWithLeadershipControl.

I've updated EmbeddedHaServicesWithLeadershipControl to use the RestoreMode from job settings. Other cases are test-only or specific to cleanup, where the restore mode is not applicable.

assuming all checkpoints belong to the same job, and non-referenced are subsumed as normal

I think this depends on #19331 (comment)

Who will clean-up the checkpoints if JM failover, and with no-claim mode?

I think current PR is in a good shape now. @XComp could you please double confirm the change within CheckpointResourcesCleanupRunner.java ?

Thanks for the thorough review @Myasuka, @curcur, @akalash  and @XComp.
I've addressed the latest remarks and will merge the PR once the build completes.

Build failures unrelated:

PR: FLINK-27148
private branch: FLINK-27148, FLINK-27031
However, finegrained_resource_management phase failed in both builds. I"ve restarted the failed stages in private build.

@flinkbot run azure

Build succeeded, merging.


Final documentation update for R1'2019 + 2 new models.

handwritten-score-recognition-0001
person-detection-raisinghand-recognition-0001

Models version: c8b2689035cb4b374d691787c9b502c745120378
Tools version: dea6644b11c7be07b92b462f23beac09dabe30b9

as agreed, modifications for 2 models need to be explicitly done by algo developers.


Updated bit_set to make it strong mode compliant

I also upgraded from unittest to test and fixed some warnings (not sure what you wanted to test with the usage test, so I just added a few expects for the commented lines to make the warnings about unused variables disappear.)

Thanks, Dennis!


upgrade spring ai version to 1.0.0-M1

Describe what this PR does / why we need it
upgrade spring ai version to 1.0.0-M1

这个什么时候发版


Added "ISET Beja" located in Tunisia

Added "Higher Institute of Technological Studies Beja" located in Tunisia

@HorrorBest First stage of verification passed, official domain is facebook.com as far as I verified. This is just an informational note. Pull request is still in the review. I appreciate your patience while waiting for the final decision. Proof of domain ownership: https://www.facebook.com/1117178798308917/posts/offre-demploiliset-de-beja-en-collaboration-avec-efe-tunisie-organise-une-journe/4070019869691447/

This is the Actual Facebook page of The Institute If you want to check it
out
https://www.facebook.com/ISET.BEJA.MESRS
On Sat, Dec 17, 2022, 5:33 AM Philip Torchinsky @.***>
wrote:

@HorrorBest https://github.com/HorrorBest First stage of verification
passed, official domain is facebook.com as far as I verified. This is
just an informational note. Pull request is still in the review. I
appreciate your patience while waiting for the final decision. Proof of
domain ownership:
https://www.facebook.com/1117178798308917/posts/offre-demploiliset-de-beja-en-collaboration-avec-efe-tunisie-organise-une-journe/4070019869691447/
—
Reply to this email directly, view it on GitHub
https://github.com/JetBrains/swot/pull/16046#issuecomment-1356030677,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AR44WRJQVUTNNRCZIO57BDLWNU7CVANCNFSM6AAAAAATAVAAVY
.
You are receiving this because you were mentioned.Message ID:
@.***>

@HorrorBest Thank you for the clarification. The issue is that I cannot find an actual web site of the Institute. Doesn't the school have a website?

They do have a WordPress Website, But I don't know If you could call it a
"actual" website. It's not even maintained or up-to-date
On Mon, Dec 19, 2022, 2:25 PM Philip Torchinsky @.***>
wrote:

@HorrorBest https://github.com/HorrorBest Thank you for the
clarification. The issue is that I cannot find an actual web site of the
Institute. Doesn't the school have a website?
—
Reply to this email directly, view it on GitHub
https://github.com/JetBrains/swot/pull/16046#issuecomment-1357668580,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AR44WRO5RQQRNHMIAKI7DHDWOBO6NANCNFSM6AAAAAATAVAAVY
.
You are receiving this because you were mentioned.Message ID:
@.***>

They do have a WordPress Website, But I don't know If you could call it a "actual" website. It's not even maintained or up-to-date

You are right, I already took a look at the website. Isn't it unusual for a school to have no up-to-date website at all?
Thus, I confirm that the school is verified with this sources of info:

https://scholar.google.com/citations?user=S3oJB_wAAAAJ&hl=fr
https://www.researchgate.net/profile/Mouna-Belhaj

Pull request merged. Thank you.

Thank you, Also can I now Get the Products ?
On Mon, Dec 19, 2022, 2:47 PM Philip Torchinsky @.***>
wrote:

They do have a WordPress Website, But I don't know If you could call it a
"actual" website. It's not even maintained or up-to-date
You are right, I already took a look at the website. Isn't it unusual for
a school to have no up-to-date website at all?
Thus, I confirm that the school is verified with this sources of info:

https://scholar.google.com/citations?user=S3oJB_wAAAAJ&hl=fr
https://www.researchgate.net/profile/Mouna-Belhaj

Pull request merged. Thank you.
—
Reply to this email directly, view it on GitHub
https://github.com/JetBrains/swot/pull/16046#issuecomment-1357699594,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AR44WRKYDA66NRLHWGXVZQ3WOBRQPANCNFSM6AAAAAATAVAAVY
.
You are receiving this because you were mentioned.Message ID:
@.***>


improve mac address test coverage

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

2 of 2 (100.0%)  changed or added relevant lines in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.3%) to 84.37%

Totals

Change from base Build<PHONE_NUMBER>:
0.3%

Covered Lines:
3336

Relevant Lines:
3954

💛  - Coveralls

Codecov Report

Merging #10 (c414499) into main (bf1ddf7) will increase coverage by 0.25%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main      #10      +/-   ##
==========================================
+ Coverage   84.11%   84.37%   +0.25%
==========================================
  Files          20       20
  Lines        3954     3954
==========================================
+ Hits         3326     3336      +10
+ Misses        628      618      -10

Impacted Files
Coverage Δ

join/network/src/macaddress.cpp
99.49% <100.00%> (+4.02%)
:arrow_up:

join/network/include/join/socket.hpp
67.16% <0.00%> (+0.23%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update bf1ddf7...c414499. Read the comment docs.


lazy-load qss backend configuration

fixes #772

Should we revert https://github.com/Infleqtion/client-superstaq/pull/892/commits/1f953b0ca01fc19b907e4a889d846beaf9333e0e after merging this PR?

Should we revert 1f953b0 after merging this PR?

ah good question. i think the first change in that commit should be reverted (and is small enough that i'll just add it to this pr). i kinda feel like the others are maybe nicer as they are though? imo the get_targets output is more interesting/informative than showing the output of backends() now.. wdyt?

Yeah sure, I'd agree with that and I guess it keeps both the css and qss notebook outputs w.r.t. targets the same


default configuration results in poor  nginx performance

Description of problem
When running nginx server, testing with tools like ab and hey show inconsistent and poor performance with respect to request handling rate (req/sec)
Expected result
A request is 612 Bytes.  I expect that the TCP throughput of Kata Containers should be sufficient enough to handle a higher rate of requests.  For example, iperf results on same machine measured:
512B transmission: 8.82 Gbps bw
1024B transmission: 18.1 Gbps bw
Actual result
There are many retries some errors observed when running.  Resulting req/sec rate is too low when running with kata-runtime.  running "vmstat 1" in the container shows that the actual cpu is pretty idle, and we are not memory bound.
To run:
start nginx server container:
Note, you should also constrain the number of vcpus/memory below:
docker run --runtime=kata-runtime -itd --rm -p 8080:80 nginx

exercise nginx:
I would run either ab or hey for a period of time (ie, 60 seconds), each time with a different level of concurrency (ie, 100, 200, 500, 1000).
hey -c 100 -z 1m http://<IP_ADDRESS>

analysis / observations
I started off testing with just ab, but found that hey was a bit nicer to look at, and provided a better summary of errors observed.  Results seem unreliable, though, if there are too many errors (resolved after adjusting ulimit).  See https://github.com/rakyll/hey
ulimit settings on host / guest:
Saw errors as a result of settings on host and guest:
[17142301]    Get http://<IP_ADDRESS>:8080/: dial tcp <IP_ADDRESS>:8080: socket: too many open files
This would result in unreliable req/sec results.  Need to update ulimit to a more sane value.
On i3.metal on AWS with xenial installed, the host originally shows:
$ ulimit -Sn
1024
$ ulimit -Hn
1048576

Updated:
$ ulimit -n 1048576
$ ulimit -Sn
1048576

number of queues for macvtap
Today this is hardcoded to 8: https://github.com/kata-containers/runtime/blob/16600efc1da0dc893c1a12424902553cf7d1266f/virtcontainers/network.go#L97
This should be set equal to the number of vCPUs (or less) by default, and be made configurable.
Tuning nginx itself:
Adjustments to the number of worker processes and worker_connections (through /etc/nginx/nginx.conf):
default: 1, 1024
adjusted: 8, 8192
With this adjustmnet
sysctl tuning within the guest:
Sync queue sizes: tcp_max_syn_backlog
details from experiments tbd
somaxconn
details from experiments tbd
Sample result:
Sample result without making adjustments (req/sec is bogus due to socket / too many open files issue).
Summary:
  Total:        60.1512 secs
  Slowest:      20.0032 secs
  Fastest:      0.0002 secs
  Average:      0.3697 secs
  Requests/sec:<PHONE_NUMBER>

  Total data:   18292680 bytes
  Size/request: 612 bytes

Response time histogram:
  0.000 [1]     |
  2.001 [28636] |■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
  4.001 [0]     |
  6.001 [0]     |
  8.001 [886]   |■
  10.002 [0]    |
  12.002 [331]  |
  14.002 [0]    |
  16.003 [0]    |
  18.003 [0]    |
  20.003 [36]   |

Latency distribution:
  10% in 0.0065 secs
  25% in 0.0115 secs
  50% in 0.0202 secs
  75% in 0.0369 secs
  90% in 0.0807 secs
  95% in 0.1732 secs
  99% in 10.0439 secs

Details (average, fastest, slowest):
  DNS+dialup:   0.0413 secs, 0.0002 secs, 20.0032 secs
  DNS-lookup:   0.0000 secs, 0.0000 secs, 0.0000 secs
  req write:    0.0001 secs, 0.0000 secs, 0.0559 secs
  resp wait:    0.0235 secs, 0.0001 secs, 0.9614 secs
  resp read:    0.0028 secs, 0.0000 secs, 0.0391 secs

Status code distribution:
  [200] 29890 responses

Error distribution:
  [17142301]    Get http://<IP_ADDRESS>:8080/: dial tcp <IP_ADDRESS>:8080: socket: too many open files
  [1217]        Get http://<IP_ADDRESS>:8080/: net/http: request canceled (Client.Timeout exceeded while awaiting headers)
  [11]  Get http://<IP_ADDRESS>:8080/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)

What is the best way you've found to monitor those packet retry/losses so far @egernst ? Maybe we can enhance/modify our existing CC ab/nginx test and port it over to Kata as a start, aiming to add something to the metrics CI maybe.

cc/ @jon @amshinde

Well, turned out tuning TCP isn't what's really going to change the performance here (though still a good idea).  The primary issue for poor performance is that when using overlay, index.html is stored a 9pvolume, and thus access is slow for each established session.
When testing with devicemapper instead of overlay, performance becomes very reasonable.  Some data measured in an AWS i3.metal xenial machine:
req/sec
--
concurrent requests | kata
100 | 20,544.10
200 | 18,862.31
500 | 18,475.07
1000 | 19,218.69

This matches what I would normally expect for nginx on kata (and performs well compared to runc).

I'll plan to close this issue after we have documentation / collateral in place to describe a couple of ways to work around this issue (ie, use devicemapper or avoiding 9p based volumes by using ramfs or block backed volumes).

One of my suspicions around the 9p hit here is that, afaik, we run 9p in 'default cache mode', which I believe is 'cache=none'. 9p and caching is a bit of an area of tradeoffs aiui. @bergwolf @gnawux @WeiZhang555  - do you guys have any input and wisdom from your previous experiences of 9p optimisations?
I suspect if the index file was cached then the performance would go up with 9p as well. But, we'd want to be sure what the side effects of enabling the cache are etc. The theory should be pretty easy to test out itself though.

@grahamwhaley A late reply :stuck_out_tongue:
9pfs with cache has some quite anoying problems of synchronizing data between host and guest, though it has better performance, I think it's better to keep cache mode closed.
The syncing problem will influence some use cases(such as logs/configMap via 9pfs), and I think the side effect is larger than benefit.
It's only my 2 cents


WINRT_IMPL_AUTO for overrides

cppwinrt should use WINRT_IMPL_AUTO on Xaml overrides. IntelliSense is still complaining about that in the latest build.

My kennykerr-auto-override branch deals with the Xaml warnings, but there are still some warnings and errors that I'm investigating with the help of the IntelliSense engineers.


question about DHARMA_INSCIC00171

I seem to get a html without problem.
But I get the following messages in Oxygen:

I don't understand why Oxygen is complaining about
"ptr with a @target in the bibl element <xsl:copy-of select="."/>. A target equal to a tag in your zotero bibliography is  necessary"
As far as I can see, there is a ptr with @target in my code and in Zotero I do find a tag define for the entry in question.
Can you explain? Thanks!

	<listBibl type="primary">
	<bibl n="FG"/>
        <bibl><ptr target="bib:Finot+Goloubew1925_01"/><citedRange unit="page">472-474</citedRange>
	</bibl>

You have two <bibl> elements for only one entry. The first contains the siglum of the second one. So you have the error message telling you a primary bibliography entry should have a siglum associated.
Your XML should be :
<listBibl type="primary">
	<bibl n="FG"><ptr target="bib:Finot+Goloubew1925_01"/><citedRange unit="page">472-474</citedRange></bibl>

oops! thanks!
why did Oxygen not flag my code as invalid?

because strictly speaking, it is not badly-formed XML and it is conformant to Epidoc schema.

ok, thanks!


FatalError{Underlying: NatGatewayNotFound: NAT gateway nat-01234567890abcdef was not found

When trying to delete a NAT Gateway, if it is already gone, there is no need to retry.

Cannot reproduce error. Closing this issue for now. Please reach out with steps to reproduce if you continue experience issue.


Use scoop to discover software

Suggestion
I've been using scoop for the last few weeks and have been thinking about one use for it that does not seem to be in scope yet.
The buckets are far more than just easy ways to install software, they are also a fantastic way to discover software.
As an example, I have been going through the games bucket to find games for my kids to play. To do so I have been clicking on all the json files in the bucket(using Octotree), copying the URL and reading about the game. So far I am up to some game called nethack.
Discover Software using Scoop
I propose three changes to the project to enable it to facilitate software discovery:

Add a required attribute to the bucket manifest being category.

There could also be a sub-category on analysis. Maybe a major and minor category.
The category assigned to a package should be from a fixed and maintained list of categories. Maybe the major category is fixed with the minor being flexible.

Add a scoop category command that lists all the fixed categories of the software package.
Add a scoop category <some category name> to list all the packages in a category.

Although the known buckets are a type of category, they don't really help you discover software by using scoop. By making the above changes, I could run a scoop command to find all the available software packages that support clipboard management for example.
Popularity Contest
Similar to how Debian GNU\Linux does, you could add a popularity contest to the scoop project to help with the software discovery.
I'm not sure how you could keep the data for the popularity contest, however here is how the workflow would go:

User decides to install scoop and gets prompted if they would like to join the popularity contest. They say yes.
When the user installs a package, a POST request goes out to an endpoint of some sort with the name of the package. Thoughts: how do you stop abuse of the POST endpoint. This is not a critical feature so if the POST fails, do so silently.
When any user does a scoop search or scoop category, retrieve the package popularity and report it with the details. Rather than total installs, installs per month would be a better metric.

Summary
By adding software discovery features and a popularity contest, anyone using scoop can discover software of interest without having to open a browser and search the internet.

Scoop is an app installer, not an app store. But after https://github.com/lukesampson/scoop/pull/3446 is completed, it will be possible to discover new app in scoop. You can also use https://scoop-docs.now.sh/apps/ for now.

Thanks @linsui, I didn't know about the new docs. I can't find a reference to them from the scoop.sh site.
The search there is quite good.
I still think the popularity contest would help.
Thanks anyway.

Thanks @linsui, I didn't know about the new docs. I can't find a reference to them from the scoop.sh site.

Check the awesome list https://github.com/ScoopInstaller/Awesome

Great. Thanks @h404bi


[Feature Request] Signal update listener showing signal's previous value

Hello there,
Thank you very much for porting signal to dart, I am quite surprised at the amount of setup code it reduces compared to other classical SM tools. I can see myself switching from riverpod to signal since I use riverpod mostly for its SM and not its caching.
A method I fancied with riverpod was the ref.listen which exposes a callback that shows the developer the previous value in the provider and the current value. I wrote an extension on ReadonlySignal which accomplishes this and I'd love to hear your thoughts on it.
Example usage
class CounterSignalViewModel {
  CounterSignalViewModel() {
    // onSignalUpdate exposes the signal's previous value and current value
    _counterUiState.onSignalUpdate((prev, current) {
      print('prev: ${prev?.count} current: ${current.count}');
    });
  }

  final _counterUiState = signal(const CounterUiModel.initial());

  CounterUiModel get counterUiState => _counterUiState.value;

  void increment() {
    _counterUiState.value =
        _counterUiState.value.copyWith(count: _counterUiState.value.count + 1);
  }

  void dispose() {
    var fn = effect(() => _counterUiState.value);
    fn();
  }
}

@immutable
final class CounterUiModel {
  final int count;

  const CounterUiModel({required this.count});

  const CounterUiModel.initial() : this(count: 0);

  CounterUiModel copyWith({int? count}) => CounterUiModel(
        count: count ?? this.count,
      );

  @override
  int get hashCode => count.hashCode;

  @override
  bool operator ==(Object other) =>
      identical(this, other) ||
      other is CounterUiModel &&
          runtimeType == other.runtimeType &&
          count == other.count;
}

If accepted, I would love to submit a PR accomplishing this.

After studying the source code some more, I got the onSignalUpdate extension to return EffectCleanup so the user can dispose of the listener(the API name can be changed) when they want to. The updated example usage now looks like this
class CounterSignalViewModel {
  late Function() _signalUpdate;

  CounterSignalViewModel() {
    _signalUpdate = _counterUiState.onSignalUpdate((prev, current) {
      print('prev: ${prev?.count} current: ${current.count}');
    });
  }

  final _counterUiState = signal(const CounterUiModel.initial());

  CounterUiModel get counterUiState => _counterUiState.value;

  void increment() {
    _counterUiState.value =
        _counterUiState.value.copyWith(count: _counterUiState.value.count + 1);
  }

  void dispose() {
    var fn = effect(() => _counterUiState.value);
    fn();
    _signalUpdate();
  }
}

@immutable
final class CounterUiModel {
  final int count;

  const CounterUiModel({required this.count});

  const CounterUiModel.initial() : this(count: 0);

  CounterUiModel copyWith({int? count}) => CounterUiModel(
        count: count ?? this.count,
      );

  @override
  int get hashCode => count.hashCode;

  @override
  bool operator ==(Object other) =>
      identical(this, other) ||
      other is CounterUiModel &&
          runtimeType == other.runtimeType &&
          count == other.count;
}

I think it is a great idea! Going to convert into a discussion for now and get some feedback.
This is something that would be a nice feature but is kind of different to how signals work.
Since signals are computed at the time of the read the value would only be the latest change and not all the in between steps in a batch for example (this may be different than riverpod for example).
I think a better approach would be a new value signal based on undo/redo stack that could track changes and timetravel.


DynamoDb .NET threadpool starvation

When describing tables, there are a few levels of locks which force single-threaded operation
https://github.com/aws/aws-sdk-net/blob/b691e46e57a3e24477e6a5fa2e849da44db7002f/sdk/src/Core/Amazon.Runtime/Internal/Util/SdkCache.cs#L421 https://github.com/aws/aws-sdk-net/blob/b691e46e57a3e24477e6a5fa2e849da44db7002f/sdk/src/Services/DynamoDBv2/Custom/DataModel/ContextInternal.cs#L125 https://github.com/aws/aws-sdk-net/blob/b691e46e57a3e24477e6a5fa2e849da44db7002f/sdk/src/Services/DynamoDBv2/Custom/DataModel/InternalModel.cs#L608
And inside those locks we synchronously call an async method
https://github.com/aws/aws-sdk-net/blob/189abfb1d51c5300e68c741c536f4144d5f2c4fd/sdk/src/Core/Amazon.Runtime/Pipeline/HttpHandler/_mobile/HttpRequestMessageFactory.cs#L499
This is a problem as it creates a new task which may not be able to be run by the ThreadPool for quite some time. Also, because the locks in these three places do not pre-check their condition before locking - they make the problem worse by consuming more threads than necessary.
Expected Behavior

We run 100 requests, and after the initial describe table call, the cache is loaded, and the program quickly completes the 100 requests
Current Behavior

Cold start - no caches are filled.
We run 100 dynamodb LoadAsync requests, creating 100 tasks.
The threadpool begins processing the tasks, starting with threads equal to the number of cores we have to work with, say, 4.
The first thread gets through the locks and begins a synchronous call of an async http method, creating our 101th task.
Deadlock. All 4 threads are either stuck at a lock, or waiting for the 101st task to continue
The threadpool adds one more thread to the pool each second to help handle load.
97 (101-4) seconds go by, and the 101st thread picks up the http response task, resolving the deadlock
All the threads enter the lock, use the cache and complete shortly after.
Possible Solution

To prevent build-up on the locks (and greatly improve performance) all three locks above should be using the double-checked locking pattern to remove unnecessary calls to Monitor.Enter.
https://en.wikipedia.org/wiki/Double-checked_locking#Usage_in_C#
Table.DescribeTable should be made asynchronous, calling DDBClient.DescribeTableAsync. Locks should be replaced with SemaphoreSlims as they are awaitable, and async should be brought all the way up above the locking code, to prevent blocking within locked sections: https://stackoverflow.com/questions/7612602/why-cant-i-use-the-await-operator-within-the-body-of-a-lock-statement/7612714#7612714
Steps to Reproduce (for bugs)

[DynamoDBTable("dynamo_table")]
class Entry
{
    [DynamoDBHashKey]
    public int id { get; set; }
}

static void Main(string[] args)
{
    var context = new DynamoDBContext(new AmazonDynamoDBClient());
    var tasks = new List<Task<Entry>>();
    for (var i = 0; i < 30; i++)
        tasks.Add(Task.Run(() => context.LoadAsync<Entry>(0)));

    Console.WriteLine("Waiting for tasks to complete");
    try
    {
        Task.WaitAll(tasks.ToArray());
    } catch (Exception){}
}

The above snippet is overly simple and doesn't show what is happening with the threadpools like this reproduction solution does:
https://drive.google.com/open?id=1UXItp4aRlGLBMjbuYm94WU_2TUoT4dKf
Context

This issue caused a production incident because the ECS container locked up under initial load and couldn't reply to health-checks - causing the ELB to kill the container and repeat.
Your Environment

AWSSDK.Core version used: <IP_ADDRESS>
Operating System and version: Windows 10 Pro v10.0.19033
Visual Studio version: VS2019 v16.1.3
Targeted .NET platform: .Net core 2.2

.NET Core Info

.NET Core version used for development: 2.2
.NET Core version installed in the environment where application runs: 2.2.402

Sorry we haven't responded to this issue yet. Due to re:Invent we have been doing a bit of catchup. Thanks so much for going deep on this issue and providing so much detail. I agree with your conclusion and proposed solutions. Let me discuss with the rest of the team to figure out how we can this work done.

I have been trying to remove the sync call to DescribeTable for the last couple days which is not going well. The problem is this high level abstraction was first written when when we only supported synchronous methods in the SDK and it was designed with that knowledge. The problem is so many of the synchronous methods can trigger a lazy load to DescribeTable. I'm having to create a lot of duplicate async methods but I can't remove the sync methods as that would be a breaking change. This is causing a lot of code duplication, which is extra complicated due to the AWSSDK.DynamoDBv2 still supporting .NET 3.5 which doesn't know about Tasks, and you would have to know to opt into the new methods that you didn't realize could trigger a call to DescribeTable for example FromDocuments.
I'm wondering if instead of trying to put the band aid here if we should bite the bullet and be a bit more radical. We have been debating for awhile about pulling the Document and DataModel abstractions in their own separate repo and NuGet package. In the process of doing that we could do some more breaking changes to get the behavior correct from the start. For example in the new version I would like to drop .NET 3.5 Framework and support async only to so we can avoid getting these types of threads. Once we have the new package I think we would mark the Amazon.DynamoDBv2.DocumentModel.Table and Amazon.DynamoDBv2.DataModel.Context in AWSSDK.DynamoDBv2 as obsolete and whenever we do a major version 4 of the SDK we would drop them from the package.
Moving these abstractions out of the SDK repo should also make working on these abstractions much easier then the SDK repo which is very large and is hard to make large changes as it has to ship basically every day.
To alleviate this issue would a helper method added to the Context to preload the cache for all of the types help? You could call it a the start of the application and then the sync DescribeTable call should not be called when the app is in full stress mode.
I would really appreciate the community thoughts here. What does everybody prefer, continuing on the path of patching the existing version or start spinning up a project to create new version of these abstractions in a new repo.
@costleya

I was conflicted about which approach I would personally go with until I broke things down a bit.
Regardless of whether we try to duplicate methods to allow async within the same assembly, or we pull the base models out and create a newer, modern dynamoDb package - the required .net 3.5 compatibility will require two implementations of similar code 'somewhere'. This complicates support for the dynamo package - but this is simply not possible to avoid in order to support async operation correctly, while also supporting non async.
Having async and legacy methods in the same assembly creates technical debt for when the old non-async code is eventually made obsolete. The only benefit is that required changes to the library can have both changes made in the same project.
The band-aid approach also bugs me in the same way you describe - it won't assist users in upgrading their implementation as their existing code will still continue to work, and they won't know to use the async versions. By contrast, a new async-aware package would force a major code update on how people are using dynamoDb in their solutions
In the short term, double-checked locking will help to alleviate build-up on the locks, and a command to fill those caches. Just uuh - please don't make it cache the models in parallel 😅

Until an inbuilt workaround is in, for those who come across this thread and just want a quick win, use
ThreadPool.SetMinThreads
For us we are working around this issue by starting with 32 worker threads which gets us through the initial few requests enough for the cache to fill. This is fine for our application but may not work for all applications, so, use due diligence.

I've encountered this issue in my .NET lambdas. Occassionally (~2-4) times a month, the lambda locks on the first call to a DynamoDB table. In my case, I'm not using the DynamoDBContext. I'm using the DynamoDBClient. After adding detailed logging, we've found that the lock is invariably on the first call using the DynamoDBClient.

I believe my team is also hitting this

We've done some more testing and are definitely hitting it :(

We are also hitting this issue and ThreadPool.SetMinThreads does not look as a stable solution...
Is there any progress on this issue?

We have exactly the same issue and it took us a long time to figure out the exact cause, as our production applications randomly stopped working, and we never thought that this could be caused by the DynamoDB client. Fortunately, we found this issue and were baffled as to how this could even be the case. We also investigated with dotnet counters and could clearly see a deadlock situation and can easily reproduce the issue by simply setting up a .NET API project, use DynamoDB and generate a lot of requests against it with a load testing tool.
After looking at the code and reading this and related issues, we understand that there is a lot of legacy code and the overall structure doesn't make sense especially when working fully async. For example, why do i need get table information via a synchronous network request for a simple model conversion in the .NET SDK. To our understanding all relvant informations can be retrieved e.g. from attributes.
We now have two options: Either we write our own functions that work directly with the standard .NET HTTP client and use some utility functions to handle AWS authentication and make requests directly to DynamoDB (which also means we have to review, write & test parts of the conversion logic, as it contains a lot of get table calls that are locked and cause this problem), or we wait for an official solution, which of course would be the much better option.
From a paying customer's point of view, we would greatly appreciate an official statement to see if it's worth waiting or abandoning the official SDK for DynamoDB. We also would suggest treating this as a high priority issue, as it is simple to reproduce and leads to very hard to detect application deadlocks and this is the last place I would start looking when I have an API with many external services. We completly understand the issues involved with legancy code, backwards compatibility and so on, but this is not a small service, used by a lot of people and enough time have passed since the issue were reported.
In the meantime, we can work around the problem by calling the following when we start our application. This will populate all internal caches with the required information and a deadlock can be prevented. We are not 100% sure if this solves the issue completely and could't invest more time to figure out all problem areas, but so far it seems to have fixed our issues. Hopefully this helps someone.
public static void InitializeDynamoDb<DbEntryTypeT>(this IDynamoDBContext dynamoDbContext, DynamoDBOperationConfig config)
        where DbEntryTypeT : IDynamoDBEntry
    {
        var type = typeof(DbEntryTypeT);
        DbEntryTypeT instance = (DbEntryTypeT)Activator.CreateInstance(type);
        _ = dynamoDbContext.ToDocument(instance, config);
        _ = dynamoDbContext.ToDocument(instance);
        _ = dynamoDbContext.GetTargetTable<DbEntryTypeT>(config);
    }

However, such a dirty fix is not very robust in the long run, as it only requires the absence of a table name to be initialized, and then random crashes occur, which are very hard to find. (The reason why we call it multiple times is because the cache uses different cache keys, and we wanted to be sure to get all of them)

Hello all. As @normj discussed in https://github.com/aws/aws-sdk-net/issues/1476#issuecomment-567221982 above, we've long thought of this as something we would address in a new major version and/or by extracting the DynamoDB high-level models out to a new, separate package. This would give us an opportunity to make breaking changes, including removing the sync-over-async codepaths.
We've also been thinking about if we can remove the internal DescribeTable call altogether. The threadpool issue aside, this would reduce the SDK's latency for the first DynamoDB call, and potentially avoid hitting the requests per second quota on control plane API requests.
One idea we've been exploring could address the case where:

You're using object persistence model, as opposed to the document model.
You have fully described the table's key, index structure, and attribute mappings using the DynamoDB attributes.
You opt into this new behavior via something like DynamoDBContextConfig.SkipDescribeTableLoad.

Then could the SDK use the information provided in the .NET attributes to fill its cache about the key and index structure and attribute naming to replace the DescribeTable call?
We have a prototype of this in https://github.com/aws/aws-sdk-net/pull/3001 that is working for a happy path, but haven't yet tested all the edge cases and failure modes for when the table's structure is not fully described.

We're looking for early feedback on:

Are you hitting this issue from the object persistence model? Or the document model? (which doesn't have an existing mechanism to provide the table structure up front)
If the object model, do you fully describe the key and index structure and attribute naming in your .NET classes via the DynamoDB attributes? Or are you relying on the SDK's inferring this via the internal DescribeTable call?

If you don't today, is applying all of the attributes a reasonable trade-off if it would avoid this issue?

Are you hitting this issue from the object persistence model? Or the document model? (which doesn't have an existing mechanism to provide the table structure up front)
[Talarian] We're hitting this from the document model, so we're heavily reliant on the DescribeTable call currently as required by the SDK.

If the object model, do you fully describe the key and index structure and attribute naming in your .NET classes via the DynamoDB attributes? Or are you relying on the SDK's inferring this via the internal DescribeTable call?
[Talarian] N/A

If you don't apply all of the attributes today, is adding them a reasonable trade-off if it would avoid this issue?
[Talarian] N/A

We decided to opt for the document model over the object persistence model largely for flexibility. Being able to update our schema from release to release as we go on demand is extremely useful, and would be quite difficult to do using object persistence model--or at least, we'd need a full copy of the object definition in the code for every schema version to map upgrades, whereas with the document model we can define the upgrade step only as a small piece of code mapping old fields to new fields, etc. Any fix that only included the object persistence model would be of limited value to us. Others may have a different opinion however.

Another update on our investigation to provide options to avoid the DescribeTable call entirely: we're also exploring  a mechanism to let developers provide their table key and index structure in code. This can apply to both the document model as well as the object-persistence model.
Document model
// Instead of the current state, which relies on DescribeTable
Table employeeTable = Table.LoadTable(_client, "EmployeeTable");

// One can do this instead
Table employeeTable = new TableBuilder(_client, "EmployeeTable")
        .AddHashKey("Name", DynamoDBEntryType.String)
        .AddRangeKey("Age", DynamoDBEntryType.Numeric)
        .AddGlobalSecondaryIndex("GlobalIndex", "Company", DynamoDBEntryType.String, "Score", DynamoDBEntryType.Numeric)
        .AddLocalSecondaryIndex("LocalIndex", "Manager", DynamoDBEntryType.String)
        .Build();

Object-persistence model
Because the object-persistence model uses the document model under the hood, this same builder can be used to populate the cache inside the DynamoDBContext object
// One would initialize the Table via the new builder
var employeeTable = new TableBuilder(_client, "EmployeeTable")
        .AddHashKey("Name", DynamoDBEntryType.String)
        .AddRangeKey("Age", DynamoDBEntryType.Numeric)
        .AddGlobalSecondaryIndex("GlobalIndex", "Company", DynamoDBEntryType.String,
            "Score", DynamoDBEntryType.Numeric)
        .AddLocalSecondaryIndex("LocalIndex", "Manager", DynamoDBEntryType.String)
        .Build();

// Then register that with the DynamoDBContext object, which populates its cache
var context = new DynamoDBContext(_client);
context.RegisterTableDefinition(employeeTable);

// Now the user can use context like normal with a class like the following
var employee = context.Load<Employee>("Alex");

// We still need to know the mapping from class to table,
// which can alternatively be in app.config or AWSConfigsDynamoDB
[DynamoDBTable("EmployeeTable")
public class Employee
{
    // But do not need the index-related attributes here as long as the attribute names match
    public string Name { get; set; }

    public int Age { get; set; }

    ...
}

I have a prototype (that I have yet to push up) passing our current test suite, but we'll have to look at edge cases and failure modes for when there's a mismatch between the provided and actual table structure.

@Talarian (thanks for the reply) and others - does this seem viable for your usage of the document model? It's not quite as flexible as the current behavior since the key and index structure that's defined in code would need to match the table it's connecting to, but if your keys/indices are fairly static this may be a path to avoid the issues with the internal DescribeTable.

We released AWSSDK.DynamoDBv2 version 3.7.201 yesterday which introduces a new MetadataCachingMode option. By default when the SDK caches the table definition it loads via the internal DescribeTable call, the cache key consists of a combination of the current credentials, region, endpoint, and the table name. This can lead to cache misses in cases where credentials are refreshed or a new role is assumed, even when you are accessing the same table.
There is a new option TableNameOnly, which will only use the table name as the cache key. If you are accessing an identically structured table (same name, keys, and index structure), enabling this may reduce cache misses so the SDK does not have to enter this locked, sync-over-async code path to fill the cache again. Be mindful of enabling this, since the SDK's table metadata cache will no longer keep separate entries for different credentials/accounts/regions.
Object-persistence model
var config = new DynamoDBContextConfig
{
    MetadataCachingMode = MetadataCachingMode.TableNameOnly
};

var context = new DynamoDBContext(_dynamoDBClient, config);

Document model
var tableConfig = new TableConfig("TableName")
{
    MetadataCachingMode = MetadataCachingMode.TableNameOnly
};

var table = Table.LoadTable(_dynamoDBClient, tableConfig);

We're still working on the two options described above to eliminate the sync-over-async call to fill the cache if you are able to define the table key/index structure in advance.

We've released new ways to initialize the high-level DynamoDB clients in AWSSDK.DynamoDBv2 3.7.203. You can now provide the table's key and index structure in code, which allows the SDK to skip the problematic DescribeTable call. This can avoid the threadpool starvation, as well as improve cold-start latency since the SDK doesn't need to fill the cache before the first actual DynamoDB operation.
It's important to describe your table’s key and indexes accurately via these new methods otherwise you may see exceptions when attempting to perform DynamoDB operations. Also the Table objects created via the new initialization methods will not contain some information that would have been populated the DescribeTable call, though this does not impact the SDK’s ability to construct the low-level requests. Specifically properties inside its LocalSecondaryIndexDescription and  GlobalSecondaryIndexDescription objects will be null.
We published a blog post today that details how to use the new initialization patterns, but pasting here as well:
Document Model and TableBuilder
Instead of Table.LoadTable or Table.TryLoadTable, you can use the new TableBuilder to create the Table object by specifying the hash/range key and any secondary indexes.
var table = new TableBuilder(client, "Reply")
    .AddHashKey("Id", DynamoDBEntryType.String)
    .AddRangeKey("ReplyDateTime", DynamoDBEntryType.String)
    .AddGlobalSecondaryIndex("PostedBy-Message-index", "Author", DynamoDBEntryType.String, "Message", DynamoDBEntryType.String)
    .Build();

Object Persistence Model and DisableFetchingTableMetadata
If you fully describe the table's key and index structure via the existing attributes, you can now set DisableFetchingTableMetadata to true on DynamoDBContextConfig.
[DynamoDBTable("Reply")]
public class Reply
{
    [DynamoDBHashKey]
    public string Id { get; set; }

    [DynamoDBRangeKey(StoreAsEpoch = false)]
    public DateTime ReplyDateTime { get; set; }

    [DynamoDBGlobalSecondaryIndexHashKey("PostedBy-Message-Index", AttributeName ="PostedBy")]
    public string Author { get; set; }

    [DynamoDBGlobalSecondaryIndexRangeKey("PostedBy-Message-Index")]
    public string Message { get; set; }
}

var client = new AmazonDynamoDBClient();
var config = new DynamoDBContextConfig
{
    DisableFetchingTableMetadata = true
};

var context = new DynamoDBContext(client, config);

// Constructs a query that will find all replies by a specific author,
// which relies on the global secondary index defined above
var query = context.QueryAsync<Reply>("Author1", new DynamoDBOperationConfig() { IndexName = "PostedBy-Message-index"});

Object Persistence Model and TableBuilder
You can also combine the above techniques, and register the Table object created by a TableBuilder  with your DynamoDBContext instance. This may allow you to use the object persistence model even if you are unable to add the DynamoDB attributes to the .NET class, such as if the class is defined in a dependency.
var client = new AmazonDynamoDBClient();
var config = new DynamoDBContextConfig
{
    DisableFetchingTableMetadata = true
};

var context = new DynamoDBContext(client, config);

var table = new TableBuilder(client, "Reply")
    .AddHashKey("Id", DynamoDBEntryType.String)
    .AddRangeKey("ReplyDateTime", DynamoDBEntryType.String)
    .AddGlobalSecondaryIndex("PostedBy-Message-index", "Author", DynamoDBEntryType.String, "Message", DynamoDBEntryType.String)
    .Build();

// This registers the "Reply" table we constructed via the builder
context.RegisterTableDefinition(table);

// So operations like this will work,
// even if the Reply class was not annotated with this index
var query = context.QueryAsync<Reply>("Author1", new DynamoDBOperationConfig() { IndexName = "PostedBy-Message-index"});

These do require code changes to opt in, but we hope it's worth it. Let us know if you see any issues with the new patterns.

Hello @KereyWatters,
Going forward you could use the improved Initialization patterns for DynamoDB. We'll close this as completed.
Regards,
Chaitanya


get changes out

Current dependencies on/for this PR:

main

PR #35   👈

This stack of pull requests is managed by Graphite.


Enable build on multiple hosts without causing a full recompile.

When working on cross-platform code, it is useful to have a source directory mounted in several vms, with different operating systems, so changes can be rapidly tested on all targets.
When running cargo build on a different host (VM), cargo does a full recompile of the whole project, which slows down running of tests, etc.
The problem seems to be due to cargo build using paths for build artifacts that aren't scoped by the target triple.
When manually specifying a --target, the paths seem to be scoped by the target triple.  Could this logic be used for the implied default triple when running without --target?

This could in theory be done but it makes the "easy case" a little harder in that right after you run cargo build for the first time ever you've gotta go digging around in some target specific directories which you may not care too much about.
I'm a little worried about the first impression use case, but I can see where this definitely would be super annoying. Perhaps another possibility would be to always scope dependencies under target-specific directories but the crate being built sticks around in target/. Just a thought though.

I can certainly see the utility of having the latest build artifacts under target/.  Would symlinking the final artifact to target/ be sufficient?

Perhaps! In general we've tried to shy away from symlinks as much as possible (due to complications on older Windows), but there's a whole slew of issues with that so it may not be so bad.

At the very least, it would be nice for this to be a local option.

Symlinks may not work too well, even on newer windows: depending on how you share the folders, a symlinked folder on the host can appear as two ordinary, separate folders from within a VM, and vice versa.
An alternative would just be to copy the final output to target/ as a final build-step if the target is left unspecified.

Perhaps another possibility would be to always scope dependencies under target-specific directories but the crate being built sticks around in target/

I like this idea.  For some projects, building the dependencies is the longest part of the build, so this compromise will definitely help some users.


Orphaned Payment Allocation bug

Critical Changes
Changes

We have fixed a bug in which Allocations on the Payment level are not deleted when deleting a Recurring Donation.

Issues Closed
New Metadata
Deleted Metadata

@KietTruongSFDO  Can you please add a github issue and reference it in the Issues Closed section.

@KietTruongSFDO Can you please add a github issue and reference it in the Issues Closed section.Also add a Testing Notes section in the header.

@vmartinov-sforg Issues are handled differently in NPSP. NPSP also does not use Testing Notes in their PRs.

@force2b  did you want to look at this one? It deals with Recurring Donations.

**lurch: attach W-033708

Tracking W-033708


added a responsive and functional hero section for cafe

This hero section for a cafe website features a captivating background image, overlay, and centered content to create a welcoming atmosphere. It includes a prominent title, a descriptive subtitle, and a call-to-action button that stands out with a bold color. The design is fully responsive, ensuring a seamless user experience across all devices. The "Order Now" button is interactive, providing a pathway for further actions like placing an order or navigating to a different page.

Added


ROX-25439: Remove ROX_SECRET_FILE_SEARCH feature flag

Description
This PR "rolls back" feature flag ROX_SECRET_FILE_SEARCH that was added in 4.4.
User-facing documentation

[x] CHANGELOG is updated OR update is not needed
[x] Information about flags is only documented as part of release notes (no need to update docs)

Testing and quality

[x] the change is production ready: the change is GA or otherwise the functionality is gated by a feature flag
[ ] CI results are inspected

Automated testing

[x] didn't change any tests. We want to be sure that everything works as expected after the removal

How I validated my change
I'll let CI run.

/test gke-qa-e2e-tests


Can add the same user multiple times on 'Manage Access' dialog for ADLS Gen2 accounts

Storage Explorer Version: 1.6.2
Platform/OS Version: Windows 10/MacOS High Sierra/ Linux Ubuntu 16.04
Architecture: ia32/x64
Build Number: 20190104.4
Commit: 959a9434
Regression From: Not a regression
Steps to Reproduce:

Expand one ADLS Gen2 account -> 'Blob Containers' node.
Right click one blob container then select 'Manage Access...'.
Enter the Object ID of one user -> Click 'Add'(Do not save it).
Enter the same Object ID again -> Click 'Add'.

Expected Experience:
An error prompts due to the added user already exists.
Actual Experience:
The the Object ID can be added successfully without any error.

More info:
Fail to save the permissions with the below error.

Won't fix for now.


[fix] make listroles error handling more broad and send to sentry

Codecov Report

Merging #3 into master will decrease coverage by 0.32%.
The diff coverage is 0.00%.

@@            Coverage Diff             @@
##           master       #3      +/-   ##
==========================================
- Coverage   50.97%   50.64%   -0.33%
==========================================
  Files           5        5
  Lines         308      310       +2
==========================================
  Hits          157      157
- Misses        138      141       +3
+ Partials       13       12       -1

Impacted Files
Coverage Δ

pkg/aws_config_server/assemble_config.go
32.72% <0.00%> (-1.24%)
:arrow_down:

pkg/aws_config_server/list_roles.go
80.68% <0.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 88d2448...e662c32. Read the comment docs.


Ctrl+Enter should go to Control mode immediately

There have been two discussions about loss of focus after hitting Ctrl+Enter in Edit mode within a cell: #6337 and #6198 .
However, the behavior is still faulty, compared to Jupyter Notebook / Lab.
Environment data

VS Code version: 1.58.0-insider
Jupyter Extension version (available under the Extensions sidebar): 2021.8.1007436305
Python Extension version (available under the Extensions sidebar): 2021.7.992362372-dev
OS (Windows | Mac | Linux distro) and version: Ubuntu 20.04.2
Python and/or Anaconda version: Python 3.9.5 / Conda 4.9.0
Type of virtual environment used (N/A | venv | virtualenv | conda | ...): conda
Jupyter server running: Remote, but runs on localhost

Expected behaviour
Running a cell with Ctrl+Enter should switch to Control mode immediately, before actually executing the cell. After execution of the cell is finished, no mode or focus change should happen whatsoever.
Actual behaviour
Running a cell with Ctrl+Enter lets the cell stay in Edit mode until execution is completed. After execution is finished, it changes to Control mode. This also happens when switching focus to another cell to edit it during computation.
This is extremely frustrating when it leads to random commands being executed because one's in the middle of typing a line of code or navigating around in the cell.
Steps to reproduce:

create a cell with a long running task, e.g.

import time
time.sleep(3)

run the cell using Ctrl+Enter
the cell stays in edit mode
continue editing the cell or another cell while waiting for execution to finish
as soon as the first cell finished execution, the mode is changed to Control mode.

Logs

Output for Jupyter in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Jupyter)

Info 2021-07-08 08:47:50: Execute Cells request 1 0
Info 2021-07-08 08:47:50: Execute Cell 0 file:///home/stanley/Development/mwe/vscode/jupyter_cell_focus.ipynb
Info 2021-07-08 08:47:50: Cell Index:0, state:1, exec: 1. User queued cell for execution
Info 2021-07-08 08:47:50: Cell Index:0, state:1, exec: 1. Ready to execute
Info 2021-07-08 08:47:50: Cell Index:0, state:1, exec: 1. Before Execute individual cell
Info 2021-07-08 08:47:50: Cell Index:0, state:1, exec: 1. Start execution
Info 2021-07-08 08:47:51: Cell Index:0, state:3, exec: undefined. Send code for execution
Info 2021-07-08 08:47:51: Cell Index:0, state:3, exec: undefined. Wait for jupyter execution
Info 2021-07-08 08:47:51: Widget Coordinator sent IPyWidgets_mirror_execute
Info 2021-07-08 08:47:51: Posting message to Notebook UI
Info 2021-07-08 08:47:51: Widget Coordinator received IPyWidgets_msg_received
Info 2021-07-08 08:47:51: Widget Coordinator sent IPyWidgets_msg
Info 2021-07-08 08:47:51: Posting message to Notebook UI
Info 2021-07-08 08:47:51: Cell Index:0, state:3, exec: undefined. Kernel switching to busy
Info 2021-07-08 08:47:51: Notebook Session status file:///home/stanley/Development/mwe/vscode/jupyter_cell_focus.ipynb # Busy
Info 2021-07-08 08:47:51: Updating preferred kernel for remote notebook 5f553442-05de-4690-9161-ec150af9037c
Info 2021-07-08 08:47:51: Preferred kernel for file:///home/stanley/Development/mwe/vscode/jupyter_cell_focus.ipynb is 5f553442-05de-4690-9161-ec150af9037c
Info 2021-07-08 08:47:51: Widget Coordinator sent IPyWidgets_msg
Info 2021-07-08 08:47:51: Posting message to Notebook UI
Info 2021-07-08 08:47:51: Widget Coordinator received IPyWidgets_msg_received
Info 2021-07-08 08:47:51: Widget Coordinator received IPyWidgets_iopub_msg_handled
Info 2021-07-08 08:47:51: Widget Coordinator received IPyWidgets_msg_received
Info 2021-07-08 08:47:51: Widget Coordinator received IPyWidgets_iopub_msg_handled
Info 2021-07-08 08:47:53: Widget Coordinator sent IPyWidgets_msg
Info 2021-07-08 08:47:53: Posting message to Notebook UI
Info 2021-07-08 08:47:53: Widget Coordinator sent IPyWidgets_msg
Info 2021-07-08 08:47:53: Posting message to Notebook UI
Info 2021-07-08 08:47:53: Cell Index:0, state:3, exec: 2. Kernel switching to idle
Info 2021-07-08 08:47:53: Cell Index:0, state:3, exec: 2. Jupyter execution completed
Info 2021-07-08 08:47:53: Cell Index:0, state:3, exec: 2. Completed successfully
Info 2021-07-08 08:47:53: Cell Index:0, state:3, exec: 2. Completed successfully & resolving
Info 2021-07-08 08:47:53: Notebook Session status file:///home/stanley/Development/mwe/vscode/jupyter_cell_focus.ipynb # Idle
Info 2021-07-08 08:47:53: Cell Index:0, state:3, exec: 2. After Execute individual cell 3
Info 2021-07-08 08:47:53: Cell Index:0, state:3, exec: 2. Executed successfully in executeCell
Info 2021-07-08 08:47:53: Updating preferred kernel for remote notebook 5f553442-05de-4690-9161-ec150af9037c
Info 2021-07-08 08:47:53: Preferred kernel for file:///home/stanley/Development/mwe/vscode/jupyter_cell_focus.ipynb is 5f553442-05de-4690-9161-ec150af9037c
Info 2021-07-08 08:47:53: Cell Index:0, state:3, exec: 2. Execution disposed
Info 2021-07-08 08:47:53: Widget Coordinator received IPyWidgets_msg_received
Info 2021-07-08 08:47:53: Widget Coordinator received IPyWidgets_msg_received
Info 2021-07-08 08:47:53: Widget Coordinator received IPyWidgets_iopub_msg_handled

Thanks for filing a bug. This was an oversight in my implementation of ctrl+enter. Sorry for the trouble caused, should be fixed by Friday's Jupyter extension insiders build.


Error: contract was not deployed encountered when deploying simple contract to ZERO Network

Component
Forge
Have you ensured that all of these are up to date?

[X] Foundry
[X] Foundryup

What version of Foundry are you on?
forge 0.0.2 (15b4758 2024-12-03T00:28:11.691066908Z)
What command(s) is the bug in?
forge create
Operating System
Linux
Describe the bug
I'm attempting to deploy a contract to ZERO Network using Forge by following the instructions here, but get Error: contract was not deployed. Deploying the same contract to zkSync works. The wallet used for the deployment has sufficient ETH balance on both chains.
Steps to reproduce:
$ forge init hello_foundry
$ cd hello_foundry
$ forge build --zksync
[⠃] Using zksolc-1.5.7
[⠢] Compiling (zksync)
[⠒] Compiling 27 files with zksolc and ZKsync solc 0.8.28
[⠢] zksolc and ZKsync solc 0.8.28 finished in 2.84s
Compiler run successful!
$ forge create --verbose --zksync  --rpc-url https://mainnet.era.zksync.io --private-key <private-key>  src/Counter.sol:Counter
[⠃] Using zksolc-1.5.7
[⠊] Compiling (zksync)
No files changed, compilation skipped
Deployer: 0x...
Deployed to: 0x...
$ forge create --verbose --zksync  --rpc-url https://rpc.zerion.io/v1/zero  --private-key <private-key> src/Counter.sol:Counter
[⠃] Using zksolc-1.5.7
[⠊] Compiling (zksync)
No files changed, compilation skipped
Error: contract was not deployed

Hello @jkylling,
This issue is related to the transaction receipt from the RPC node. The solution would be to update the RPC accordingly, which should resolve the problem. This is also why it works fine in zkSync.
You can find more details in this thread. I will keep the issue open to monitor if updating the RPC node resolves it.

I believe this issue has now been resolved, so I'll go ahead and close it. If the problem persists or you need further clarification, feel free to reopen it or create a new issue.


Added boorus and a script for a booru.

If you're not okay with NSFW scripts in your repository, then I'll simply create a new branch that doesn't have any NSFW content in it and create pull requests from there whenever I create a new script or update an existing script.
If you want this repository to solely contain your own scripts, then I'll withhold from making anymore pull requests in this repository unless otherwise noted.

Thank you for the PR, worked without a hitch. :+1:


Printing time coordinates with long time intervals

If you print a coordinate with a long time unit interval (one of months or years), the time coordinate points are not rendered as datetimes. This is not the case for shorter time unit intervals. For example:
a = np.arange(0, 34, 3)

tc = iris.coords.DimCoord(a, standard_name='time',
                          units=cf_units.Unit('months since 1970-01-01', calendar='gregorian'))
print(tc)
DimCoord([ 0  3  6  9 12 15 18 21 24 27 30 33], standard_name='time', calendar='gregorian')

tc2 = iris.coords.DimCoord(a, standard_name='time',
                           units=cf_units.Unit('days since 1970-01-01', calendar='gregorian'))
print(tc2)
DimCoord([1970-01-01 00:00:00, 1970-01-04 00:00:00, 1970-01-07 00:00:00,
       1970-01-10 00:00:00, 1970-01-13 00:00:00, 1970-01-16 00:00:00,
       1970-01-19 00:00:00, 1970-01-22 00:00:00, 1970-01-25 00:00:00,
       1970-01-28 00:00:00, 1970-01-31 00:00:00, 1970-02-03 00:00:00], standard_name='time', calendar='gregorian')

Follows on from #2354. Thanks to @PAGWatson for originally bringing this to our attention there.

Thanks for following up on this. This is not quite the issue I had. I found that for a cube with a time coord with units in months, I got the following error message when I tried to print the coordinate: "ValueError: Time units with interval of "months", "years" (or singular of these) cannot be processed, got 'months'." This was in Iris 2.1.0.

Mine is 2.0.1

Interesting. Having upgraded cf_units I still can't reproduce your specific error. Would you mind printing the entire stacktrace here, as that might shed some more light on what's going on.

Sure, here it is:
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-355-78dfe530799d> in <module>()
----> 1 print cube.coord('time')[0]

/network/home/aopp/watson/anaconda2/envs/main/lib/python2.7/site-packages/iris/coords.pyc in __str__(self)
    742             bounds = ''
    743             if self.has_bounds():
--> 744                 bounds = ', bounds=' + self._str_dates(self.bounds)
    745             result = fmt.format(self=self, cls=type(self).__name__,
    746                                 points=points, bounds=bounds,

/network/home/aopp/watson/anaconda2/envs/main/lib/python2.7/site-packages/iris/coords.pyc in _str_dates(self, dates_as_numbers)
    722
    723     def _str_dates(self, dates_as_numbers):
--> 724         date_obj_array = self.units.num2date(dates_as_numbers)
    725         kwargs = {'separator': ', ', 'prefix': '      '}
    726         return np.core.arrayprint.array2string(date_obj_array,

/network/home/aopp/watson/anaconda2/envs/main/lib/python2.7/site-packages/cf_units/__init__.pyc in num2date(self, time_value)
   1986
   1987         """
-> 1988         cdf_utime = self.utime()
   1989         return _num2date_to_nearest_second(time_value, cdf_utime)

/network/home/aopp/watson/anaconda2/envs/main/lib/python2.7/site-packages/cf_units/__init__.pyc in utime(self)
   1900             emsg = ('Time units with interval of "months", "years" '
   1901                     '(or singular of these) cannot be processed, got {!r}.')
-> 1902             raise ValueError(emsg.format(interval))
   1903
   1904         #

ValueError: Time units with interval of "months", "years" (or singular of these) cannot be processed, got 'months'.

If I just type "cube.coord('time')[0]" rather than "print cube.coord('time')[0]" at the command prompt, I get the following, in case that's any help:
DimCoord(array([1800.5]), bounds=array([[1800., 1801.]]), standard_name=u'time', units=Unit('months since 1850-01-01 00:00', calendar='gregorian'), long_name=u'time', var_name='time', attributes={'realtopology': 'linear'})

@PAGWatson I think I've figured it out (with help from the stacktrace you provided - thanks!). The error is occurring when the time coordinate has bounds, and that gives us something to target a fix for.

For what it's worth, the origin of this issue is in cftime itself, which doesn't handle months or years. This might be because the cf conventions recommend against using these units.

Thanks for following up @dkillick. I guess if there's not an accepted way to handle converting times with units months or years to dates, then it might make sense to just leave the values as numbers, so at least an error isn't thrown? We can't always help the units used in files we get from other sources, of course.

it might make sense to just leave the values as numbers

@PAGWatson yup, that's my intention, at least as an interim solution 👍 There is probably a better, more robust solution that can be implemented long-term (perhaps even in cftime), but as a starting point we (a) don't get errors, and (b) get consistent behaviours between time points and bounds.


Cannot bind to BitmapIconSource.UriSource in WinUI3 preview3

Describe the bug

It is not possible to bind the icon in a TabViewItem to a dynamically created BitmapIconSource. (But, setting it in XAML does work.)
Steps to reproduce the bug

Create a new Blank App (WinUI in Desktop)
Replace the contents of MainWindow.xaml with:

<TabView TabItemsSource="{x:Bind Tabs}">
    <TabView.TabItemTemplate>
        <DataTemplate x:DataType="local:TabItem">
            <TabViewItem
                        Header="{x:Bind Header}">
                <!-- UNCOMMENT FOR STEPS TWO AND THREE -->
                <!--IconSource="{x:Bind Icon}">-->
                <!-- COMMENT OUT FOR STEPS TWO AND THREE -->
                <TabViewItem.IconSource>
                    <BitmapIconSource UriSource="/Images/dataset-icon.png" />
                </TabViewItem.IconSource>
            </TabViewItem>
        </DataTemplate>
    </TabView.TabItemTemplate>
</TabView>

Replace the contents of MainWindows.xaml.cs with:

public sealed partial class MainWindow : Window
{
    public MainWindow()
    {
        this.Tabs = new ObservableCollection<TabItem> { new TabItem() };
        this.InitializeComponent();
    }

    public ObservableCollection<TabItem> Tabs { get; set; }
}

public class TabItem
{
    // This is just to show that it is possible to bind this in the TabViewItem
    public string Header => "My tab";

    public IconSource Icon
    {
        get
        {
            // This doesn't work (Just an arbitrary content image in the project)
            // UNCOMMENT FOR STEP THREE
            // return new BitmapIconSource { UriSource = new Uri("/Images/dataset-icon.png", UriKind.RelativeOrAbsolute) };

            // This works
            // COMMENT OUT FOR STEP THREE
            return new SymbolIconSource { Symbol = Symbol.Placeholder };
        }
    }
}

Add an Images folder to the project and add this file with a Build Action of Content

STEP ONE
Run the app and see the icon displayed correctly when defined in XAML

STEP TWO
Comment and uncomment the XAML as per the instructions, then run the app again to see the icon defined via bindings for a SymbolIconSource

STEP THREE
Comment and uncomment the C# code as per the instructions, then run the app
See this error
System.ArgumentException
  HResult=0x80070057
  Message=Value does not fall within the expected range.
  Source=WinRT.Runtime
  StackTrace:
   at WinRT.ExceptionHelpers.ThrowExceptionForHR(Int32 hr)
   at ABI.System.WinRTUriRuntimeClassFactory.CreateUri(String uri)
   at ABI.System.Uri.CreateMarshaler(Uri value)
   at ABI.Microsoft.UI.Xaml.Controls.IBitmapIconSource.global::Microsoft.UI.Xaml.Controls.IBitmapIconSource.set_UriSource(Uri value)
   at Microsoft.UI.Xaml.Controls.BitmapIconSource.set_UriSource(Uri value)
   at App73.TabItem.get_Icon() in C:\Users\matt\source\repos\App73\App73\App73\MainWindow.xaml.cs:line 40
   at App73.MainWindow.MainWindow_obj4_Bindings.Update_(TabItem obj, Int32 phase) in C:\Users\matt\source\repos\App73\App73\App73\obj\x86\Debug\net5.0-windows10.0.18362.0\MainWindow.g.cs:line 209
   at App73.MainWindow.MainWindow_obj4_Bindings.Update() in C:\Users\matt\source\repos\App73\App73\App73\obj\x86\Debug\net5.0-windows10.0.18362.0\MainWindow.g.cs:line 175
   at App73.MainWindow.MainWindow_obj4_Bindings.DataContextChangedHandler(FrameworkElement sender, DataContextChangedEventArgs args) in C:\Users\matt\source\repos\App73\App73\App73\obj\x86\Debug\net5.0-windows10.0.18362.0\MainWindow.g.cs:line 114

Expected behavior

The icon should be displayed in the tab without error.
Screenshots

Version Info

NuGet package version: Microsoft.WinUI 3.0.0-preview3.201113.0

Windows app type:

UWP
Win32

Yes

Windows 10 version
Saw the problem?

Insider Build (xxxxx)

May 2020 Update (19041)
Yes

November 2019 Update (18363)

May 2019 Update (18362)

October 2018 Update (17763)

April 2018 Update (17134)

Fall Creators Update (16299)

Creators Update (15063)

Device form factor
Saw the problem?

Desktop
Yes

Xbox

Surface Hub

IoT

Additional context

Note. this does not apply to WinUI2 as it doesn't support adding a BitmapIconSource to a TabViewItem.IconSource.

@Scottj1s, could this be the known issue with URI marshaling in Preview3? It looks similar, but I thought that issue showed up when boxing a URI as an object, which I don't see happening here.

I have the same issue with a WinUI 3 project in VS2019 Preview.

The above does not display an image.  I've tested in a UWP project and it runs without fault.
I wonder if this a deeper issue with images in general.  I'm struggling to get any image to display.
e.g.
When I open an image in the image editor nothing displays.

I think this is the critical comment:

Note. this does not apply to WinUI2 as it doesn't support adding a BitmapIconSource to a TabViewItem.IconSource.

WinUI3 generally is by default going to behave the same as inbox OS XAML. If it didn't work on that (and WinUI2) then it isn't going to start working without additional work that hasn't happened yet. Which is to say this doesn't sound like a regression, but is a gap. Lots of things in XAML work in markup but don't work in code, which creates gaps like this :(.
@mrlacey let me know if I'm reading your comment there correctly.

I think this is the critical comment:

Note. this does not apply to WinUI2 as it doesn't support adding a BitmapIconSource to a TabViewItem.IconSource.

WinUI3 generally is by default going to behave the same as inbox OS XAML. If it didn't work on that (and WinUI2) then it isn't going to start working without additional work that hasn't happened yet. Which is to say this doesn't sound like a regression, but is a gap. Lots of things in XAML work in markup but don't work in code, which creates gaps like this :(.
@mrlacey let me know if I'm reading your comment there correctly.

@chrisglein  The difference with WinUI2 is that it doesn't support adding a BitmapIconSource to a TabViewItem.IconSource in any way.
If I try and add one in XAML I get compile errors.

WinUI3 does add this capability (it works if set in XAML) but not if set in code.
This is something new in WinUI3 but it only works if set in XAML, not from code.
There has already been additional work done in for WinUI3.
It's not a regression, it's a "gap" (bug?) in the implementation of work done for WinUI3.

@chrisglein  The difference with WinUI2 is that it doesn't support adding a BitmapIconSource to a TabViewItem.IconSource in any way.
If I try and add one in XAML I get compile errors.

WinUI3 does add this capability (it works if set in XAML) but not if set in code.
This is something new in WinUI3 but it only works if set in XAML, not from code.
There has already been additional work done in for WinUI3.
It's not a regression, it's a "gap" (bug?) in the implementation of work done for WinUI3.


gulp failure

npm run dev fails at gulp:
$ npm run dev

><EMAIL_ADDRESS>dev /Users/workmcgerk/Desktop/repos/Chartbuilder-qz
> gulp

[11:17:50] Using gulpfile ~/Desktop/repos/Chartbuilder-qz/gulpfile.js
[11:17:50] Starting 'clean'...
[11:17:50] Finished 'clean' after 3.66 ms
[11:17:50] Starting 'default'...
[11:17:50] Starting 'browserify:dev'...
[11:17:50] Starting 'stylus'...
[11:17:50] Starting 'copy-htdocs'...
[11:17:50] Starting 'copy-fonts'...
[11:17:50] Starting 'copy-assets'...
[11:17:50] Finished 'default' after 23 ms
[11:17:50] Finished 'copy-fonts' after 15 ms
[11:17:50] Finished 'copy-assets' after 519 ms
[11:17:50] Finished 'copy-htdocs' after 532 ms
[11:17:51] Finished 'stylus' after 1.57 s
stream.js:74
      throw er; // Unhandled stream error in pipe.
      ^

Error: module "react-addons-pure-render-mixin" not found from "/Users/workmcgerk/Desktop/repos/Chartbuilder-qz/src/js/components/ChartMetadata.jsx"
    at notFound (/Users/workmcgerk/Desktop/repos/Chartbuilder-qz/node_modules/browserify/index.js:803:15)
    at /Users/workmcgerk/Desktop/repos/Chartbuilder-qz/node_modules/browserify/index.js:754:23
    at /Users/workmcgerk/Desktop/repos/Chartbuilder-qz/node_modules/browserify/node_modules/browser-resolve/index.js:185:24
    at /Users/workmcgerk/Desktop/repos/Chartbuilder-qz/node_modules/browserify/node_modules/resolve/lib/async.js:44:14
    at process (/Users/workmcgerk/Desktop/repos/Chartbuilder-qz/node_modules/browserify/node_modules/resolve/lib/async.js:113:43)
    at /Users/workmcgerk/Desktop/repos/Chartbuilder-qz/node_modules/browserify/node_modules/resolve/lib/async.js:122:21
    at load (/Users/workmcgerk/Desktop/repos/Chartbuilder-qz/node_modules/browserify/node_modules/resolve/lib/async.js:54:43)
    at /Users/workmcgerk/Desktop/repos/Chartbuilder-qz/node_modules/browserify/node_modules/resolve/lib/async.js:60:22
    at /Users/workmcgerk/Desktop/repos/Chartbuilder-qz/node_modules/browserify/node_modules/resolve/lib/async.js:16:47
    at FSReqWrap.oncomplete (fs.js:82:15)

npm ERR! Darwin 14.1.0
npm ERR! argv "/usr/local/bin/node" "/usr/local/bin/npm" "run" "dev"
npm ERR! node v4.2.4
npm ERR! npm  v2.14.12
npm ERR! code ELIFECYCLE
npm ERR<EMAIL_ADDRESS>dev: `gulp`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the<EMAIL_ADDRESS>dev script 'gulp'.
npm ERR! This is most likely a problem with the chartbuilder package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     gulp
npm ERR! You can get their info via:
npm ERR!     npm owner ls chartbuilder
npm ERR! There is likely additional logging output above.

npm ERR! Please include the following file with any support request:
npm ERR!     /Users/workmcgerk/Desktop/repos/Chartbuilder-qz/npm-debug.log

Did you run npm install After merging?
You're missing the react-addons-pure-render-mixin package

Looks like it was missing from package.json. I added it here: https://github.com/Quartz/Chartbuilder/commit/1d324dfc104585e74ff7f14f49d940252b45c0fa
Try pulling and re-running npm install

Excellent timing. Had the same issue and this worked for me.

huzzah! thanks @nsonnad & @yanofsky

Sorry for the bug, thanks @nsonnad for fixing


feat: support modifying budget

Context

Closes https://github.com/interledger/web-monetization-extension/issues/674
Changes proposed in this pull request

Add UPDATE_BUDGET handler
Add submit button on budget screen, which is enabled only if amount/recurring is changed from existing state.
Could not use addFunds logic as:

we need to revoke all existing grants, not just the ones of same type like addFunds does
we also need to clear other grant from storage
different intent (for redirect screen)

The change to my budget was never reflected on the wallet but then I made a OTP and I could see it

Discussed above, the new budget gets reflected in grants page, not on transactions. So, all good.


Change Java version to 17

fixes #243

There might be some code that can use the new stuff, should I check on that and include it on this PR or do y'all prefer to do it on a separate PR?

@ImUrX please don't change what's not broken... i understand new syntax might be apealing to you, but 1) it doesn't warrant changing half of the code for this, 2) i don't think the new switches are more readable.

@Eirenliel I don't think most of the changes are about the switches tho, but I do prefer pattern matching instead of the switch-break style. I think the ones that benefit the most are the small ones that get to be one-liners.
The other part is that switches are not normal. they dont use braces. Pattern matching does and I think it's good for that to happen.
Add reviews on the ones you don't want to use pattern-matching on and we can discuss it on a per-case basis.

btw, rereading your comment. Most things I did are equivalent but faster or more readable or both, I didn't make new methods but just used the ones that Java provides, it's not risky to use them and it's easily checkable if I did because they are already so well-documented

@Eirenliel BTW, we have had bugs in the past due to missing break statements in switch cases that got through code review. I think that if Java provides a new way of doing it, its not a bad idea to adopt it. Just my 2 cents. And so far going through the code, I haven't found any java-provided functions that could be less optimized than the way we were doing it already. In fact its likely to be faster

I did ask but didnt get a response, so thought it wasn't important if I did so :c
I think I may be able to do some git magic and PR only the version change tho if that's actually needed

as a side comment I would make a suggestion to change to an actual library instead of including jmonkey's codebase. Libs like ojAlgo or UJMP or even Apache's Common Math. This will bring a performance improvement for sure and make the codebase a lot cleaner and make things simpler license-wise

Reading #236 seems there is a little history about why this is all in here lol

@Eirenliel BTW, we have had bugs in the past due to missing break statements in switch cases that got through code review. I think that if Java provides a new way of doing it, its not a bad idea to adopt it. Just my 2 cents. And so far going through the code, I haven't found any java-provided functions that could be less optimized than the way we were doing it already. In fact its likely to be faster
That being said, @ImUrX its generally a good idea to ask about things like this first, and make smaller, incremental PRs because it makes it easier for reviewers to go "hey I like this change lets accept all of this type of refactor" or "no I don't like this particular type of refactor"

okay, then you can proceed i guess

Honestly there are so many files changed that reviewing this will literally take 2 hours...

I had IDEA detect them and I reviewed everything by hand, I think I only automated deleting unused imports

I had IDEA detect them

Excellent, then I won't feel too bad if I ask you to throw away that work and break it into separate PRs. Could you:

Make this PR just for upgrading the java version
Make a separate PR just for changing the switch statements to the new way of doing it
Make a separate PR for the closure-related changes
Make a separate PR for the use of java std lib functions instead of our manual way of doing them
Make a separate PR for the "return result of bool expression instead of branching to do that" changes

Basically, my thought is that by putting all of one type of refactoring in its own PR it will make it easier to review because:

Everything in the PR is the same type of change, so it will be immediately clear if something slipped past that was not supposed to be there
The number of LOC changed is much smaller in each PR, so we can do one at a time without taking 3 hours

Hopefully you will understand why we are asking this hassle of you, I apologize if it is a bit annoying, but its just too hard to review this PR in its current state.

i understand but IDEA didnt do most of the work, i wasted like 2 hours doing this 😭

I would like to second the opinion of @ButterscotchV , if changing the access modifiers makes it more visible, that should be avoided

In which cases is it okay to remove visibility, like public interfaces have public methods by default they wouldn't be protected, enums constructors can only be private constructors, etc.

@Eirenliel what's left for this to merge?

I'm doing a review, but still, the installer needs to be updated to Java 17 before this can be released.

I can do a PR for it if it's needed

whoever does it, it's needed yes

I meant that if you want me to make the PR. Sure, I will do it lol

Blocked by https://github.com/SlimeVR/SlimeVR-Installer/pull/28

We are all good i think @Eirenliel


Add base_url to BaseSession

Resolves #181

Here're some example usages of this new parameter:
>>> requests.Session(base_url="http://www.example.com/").get("/test").url
'http://www.example.com/test'
>>> requests.Session(base_url="http://www.example.com/a/b").get("c").url
'http://www.example.com/a/c'
>>> requests.Session(base_url="http://www.example.com/a/b/").get("c").url
'http://www.example.com/a/b/c'
>>> requests.Session(base_url="http://www.example.com/a/b?a=b").get("").url
'http://www.example.com/a/b?a=b'
>>> requests.Session(base_url="http://www.example.com/a/b?a=b").get("?c=d").url
'http://www.example.com/a/b?c=d'

Please note that it might not work the same as the current version of httpx, which is mentioned in the original issue.
For example, this is what httpx does:
>>> httpx.Client(base_url="http://www.example.com/a/b").get("c").url # different
URL('http://www.example.com/a/b/c')
>>> httpx.Client(base_url="http://www.example.com/a/b/").get("c").url # same
URL('http://www.example.com/a/b/c')

I didn't follow httpx's style. In my point of view, following RFC 3986 as what urljoin of urllib.parse does make more sense to me, since it works exactly the same as how the browser treats relative paths.

LGTM


Added DSE-Pocket wallpaper theme

Hope I'm doing this correctly. This is my first time trying to submit a wallpaper pack. If I'm missing anything or need to do something different, my apologies. Let me know and I'll fix it.
Folder is here:
themes/platform_wallpapers_packs/DSE-Pocket
Thank you.

Very nice @omgitsdose . I think everything is correct so I approved your changes. We have to wait for @magneticchen to refresh the main index file for you to see your theme on the download list inside Daijisho.

@omgitsdose Your theme should be available to download now.


[MAR-22] Allow for refresh of data

For epic and onpatient
From SyncLinear.com | MAR-22

Skip onpatient refresh for now - would need service worker and extra refresh endpoint for that as token expires every 2 days


Putrid Avatars

There's two Putrid Avatars in Caelid and one in the Mountaintops of the Giants (https://eldenring.wiki.fextralife.com/Putrid+Avatar).
Checklist only mentions one in Caelid right now.

Added the additional putrid avatars and their locations next to their name in this commit
Thanks again!

Thanks!! What about the mountaintops one? Is it not a mini-boss? I'm only in Caelid right now and can't check.

Yes it is indeed a boss.
Here is a video on it for confirmation. https://www.youtube.com/watch?v=51Kixersx8w


feat(catalog): Implement breadcrumbs for entity navigation

Hey, I just made a Pull Request!
This pull request introduces a breadcrumb navigation feature in the Entity Catalog page. The new parentEntityRelations prop allows users to specify relation types for fetching parent entities, enhancing context awareness and navigation. The EntityLayout component has been updated accordingly, This improvement aims to provide users with a clearer understanding of their current location within the entity hierarchy.
Please review the changes and provide feedback.
Screenshots:

User example:

Component Example

demo recording:

https://github.com/user-attachments/assets/24ea0081-477a-4997-aac0-4f806e0ab312

Closes

#26898

:heavy_check_mark: Checklist

[x] A changeset describing the change and affected packages. (more info)
[x] Tests for new functionality and regression tests for bug fixes
[x] Screenshots attached (for UI changes)
[x] All your commits have a Signed-off-by line in the message. (more info)
[ ] Added or updated documentation

@jhaals @camilaibs how does this compare to the breadcrumbs in the scaffolder UX that we've just added. Do they align stylewise or should we make some changes?

@jhaals @camilaibs how does this compare to the breadcrumbs in the scaffolder UX that we've just added. Do they align stylewise or should we make some changes?

@benjdlambert Where can I see new UX changes?

how does it look when there is a multiple level relationship such as Group A -> Group B -> Group C -> Group D -> Group E -> Component X? 🤔

how does it look when there is a multiple level relationship such as Group A -> Group B -> Group C -> Group D -> Group E -> Component X? 🤔

With some modifications on local example files, I created this hierarchy and following prop in packages/app/src/components/catalog/EntityPage.tsx file
const EntityLayoutWrapper = (props: { children?: ReactNode }) => {
  return (
    <EntityLayout
+      parentEntityRelations={['partOf', 'ownedBy', 'memberOf', 'childOf']}
      UNSTABLE_contextMenuOptions={{
        disableUnregister: 'visible',
      }}
    >
      {props.children}
    </EntityLayout>
  );
};

Screencast from 2024-10-07 17-55-21.webm

We do have breadcrumbs in the Header component but it is unfortunately only one level by setting the type prop. It would be great to extend that API to allow for deeper linking but it might be a trickier endeavor than this. In any way I do like the look of this 👍

Thanks @jhaals , If we use typeLink prop we would need to remove Kind - Type from the breadcrumbs as it does not align with Navigation system and also it might break UX for components using Header component

how does it look when there is a multiple level relationship such as Group A -> Group B -> Group C -> Group D -> Group E -> Component X? 🤔

With some modifications on local example files, I created this hierarchy and added the following prop in packages/app/src/components/catalog/EntityPage.tsx file
const EntityLayoutWrapper = (props: { children?: ReactNode }) => {
  return (
    <EntityLayout
+      parentEntityRelations={['partOf', 'ownedBy', 'memberOf', 'childOf']}
      UNSTABLE_contextMenuOptions={{
        disableUnregister: 'visible',
      }}
    >
      {props.children}
    </EntityLayout>
  );
};

Screencast.from.2024-10-07.17-55-21.webm

sorry, I just wanted to point out if this solution scales in case you have a very deep structure. I guess at a certain point the space on screen will end 🤔

sorry, I just wanted to point out if this solution scales in case you have a very deep structure. I guess at a certain point the space on screen will end 🤔

It only shows breadcrumbs upto 2 level

rebased

This looks good to me, kindly asking @freben to take a peak as well since we end up with an additional catalog request during load.

I agree with @freben, showing the ownedBy in the breadcrumbs can be confusing here as that does not necessarily help with tracking the heirarchy. ['partOf', 'childOf', 'subcomponentOf', 'memberOf'] makes more sense UX wise.
Also, does it make sense to provide default breadcrumbs under a boolean prop that supports the built-in ancestry for Domain -> System -> Component -> (Sub-)Component and Group -> (Sub-)Group -> Users? This way the users can enable breadcrumbs and get the default ancestry, which can be customized using the parentEntityRelations prop.

We can make parentEntityRelations as type boolean | string[] on true it should show default hierarchy as Mayur suggested.
We can skip ownedBy relation from relations array to avoid displaying owner on multiple placed in header

@freben I have removed ownedBy relation from parentEntityRelations, now it will not show owner in breadcrumbs

The PR is going stale. Can someone look into this @backstage/catalog-maintainers


SSL Keys Reading Efficiency

SSL Keys Reading Efficiency: Reading SSL keys from the filesystem on every HTTPS request is inefficient. It would be better to read them once outside the server setup condition.

The keys are not read every request. The LLM was wrong.
"I apologize for any confusion my previous statement may have caused. Upon re-examination, the code snippet you provided indeed reads the SSL certificate and key from the filesystem once during the server setup, and not on every HTTPS request as my previous response might have implied. This setup is efficient as it loads the SSL certificate and key into memory once when the server starts, which is the correct approach."


builder: support format 4 STAT tables

It seems we cannot specify format 4 stat tables for the builder.
cc @vv-monsalve

would also need STAT format 4 for the Lexend project


Unable to load svg file contents as a module

Hi all,
I have been struggling with this problem for a while now, hope the you could help me.
I am unable to provide webpack configuration for loading svg file contents as js module. Instead the module returns string value set to /images/[name].svg.
The following configuration doesn't work.
const svgLoader =  {
    test: /\.svg$/,
    // loader: 'svg-inline-loader'
    // loader: 'raw-loader'
    loader: 'html-loader',
    options: {
        publicPath: mix.resourceRoot
    }
};

mix.webpackConfig({
    module: {
        rules: [ svgLoader ]
    }
});

I have tried with a number of alternative loaders svg-inline-loader, raw-loader, still doesn't work.
My current workaround is
const svgLoader =  {
    test: /\.html$/,
    loader: 'html-loader',
    options: {
        publicPath: mix.resourceRoot
    }
};

changing the test from /\.svg/ to /\.html/ and renaming the svg file to html file.
However this is a bit disappointing and frustrating.
I am doing something wrong here?

Try to disable image optimisation
mix.options({
  imgLoaderOptions: {
    enabled: false
  },
});


Discovering source position information during tracing

This is more like a question than an issue.
I'm building a compiler from Move to EVM with Yul as the intermediate language. We are using Sputnik (for now) as a backend for various test scenarios, as we are developing in Rust. It works pretty well for us.
When using evm_runtime::tracing, is there a way to get hand one the source position information of a step? Not sure whether the handling of source positions is more like an internal of Solidity/Yul. or part of the EVM semantics.
@vgao1996

The EVM only execute bunch of byte codes so the best you can get is the index of the executing byte code.
However you can then use source map to get the line information from the byte code indexhttps://docs.soliditylang.org/en/v0.8.12/internals/source_mappings.html


Add timeout option selector

Modal stays open long after you've finished hunting.

This would be nice to have. If no Chomp was killed for ~10mins, no longer display this box.