Opposing results when comparing to transformers interpret

ferret version: 0.4.0
Python version: 3.9.2
Operating System: Linux Debian

Description
When comparing your feature attribution scores of the explanation provided by Integrated Gradients (plain) with the ones by the transformers_interpret library (MultiLabelClassificationExplainer), I get significantly different results. For example, a token may have a high score of 0.5 with transformers_interpret, but is negatively attributed with ferret.
Why could that be ?
Of course, I tested this on the same conditions for both transformers_interpret and ferret (e.g.: pretrained local multi-label BertForSequenceClassification, bert-base-german-cased tokenizer, same sample)
What I Did

transformers_interpret:

cls_explainer = MultiLabelClassificationExplainer(model, tokenizer, custom_labels=labels)
word_attrib = cls_explainer(<SAMPLE>)
pred = cls_explainer.predicted_class_name
print(word_attrib[pred])

ferret:

bench = Benchmark(model, tokenizer)
score = bench.score(<SAMPLE>)
metr = bench.explain(sent, target=target)[4] ### IG (plain) ###
print(metr.scores)

Hi, thank you for reaching out. Can you give me the version of transformer interpret you are using and the sentence you are explaining so that I can setup a quick colab?

Hi, thank you for reaching out. Can you give me the version of transformer interpret you are using and the sentence you are explaining so that I can setup a quick colab?
EDIT: also, I notice you are specifying 0.8.0 as the ferret version, but that cannot be correct. Are you using the latest v0.4.1?

Hey, thank you for your fast answer! Yes I'm sorry, the version of ferret should have stated: 0.4.0 and transformers_interpret is 0.9.6. I talked to my co-worker about this and it seems this is the case for any sample, e.g.:  "Röntgenologisch wird kein V.a. eine Krankheit gestellt." where "eine" receives a -0.04 score with ferret's plain IG & 0.48 with transformers_interpret

Perfect! Can you also tell me which model checkpoint you are using? (if it's a private model, I can try a similar public one if you point me to it).
In the meantime, you could try to set the normalization false in the explain method. By default, we apply L1 normalization across all tokens to make attribution scores comparable across different explainers.
metr = bench.explain(sent, target=target, normalize_scores=False)[4] ### IG (plain) ###

Hey,
please excuse my late reply, but the checkpoint I'm using is the 10000th one of the pretrained BertForSequenceClassification (bert-base-german-cased) with the following configuration:
{
  "_name_or_path": "bert-base-german-cased",
  "architectures": [
    "BertForSequenceClassification"
  ],
  "attention_probs_dropout_prob": 0.1,
  "classifier_dropout": null,
  "hidden_act": "gelu",
  "hidden_dropout_prob": 0.1,
  "hidden_size": 768,
  "id2label": {
    "0": "LABEL_0",
    "1": "LABEL_1",
    "2": "LABEL_2",
    "3": "LABEL_3",
    "4": "LABEL_4",
    "5": "LABEL_5",
    "6": "LABEL_6",
    "7": "LABEL_7",
    "8": "LABEL_8",
    "9": "LABEL_9",
    "10": "LABEL_10"
  },
  "initializer_range": 0.02,
  "intermediate_size": 3072,
  "label2id": {
    "LABEL_0": 0,
    "LABEL_1": 1,
    "LABEL_10": 10,
    "LABEL_2": 2,
    "LABEL_3": 3,
    "LABEL_4": 4,
    "LABEL_5": 5,
    "LABEL_6": 6,
    "LABEL_7": 7,
    "LABEL_8": 8,
    "LABEL_9": 9
  },
  "layer_norm_eps": 1e-12,
  "max_position_embeddings": 512,
  "model_type": "bert",
  "num_attention_heads": 12,
  "num_hidden_layers": 12,
  "pad_token_id": 0,
  "position_embedding_type": "absolute",
  "problem_type": "single_label_classification",
  "torch_dtype": "float32",
  "transformers_version": "4.23.0.dev0",
  "type_vocab_size": 2,
  "use_cache": true,
  "vocab_size": 30000
}

Hey, sorry for the long radio silence.
In your last code example, normalization is still happening, so that might be an issue. Anyway, another different factor I see is the baseline used for computing Integrated Gradients. We are currently using a [CLS] + token count * [PAD] + [SEP] scheme and that might be different than what transformer interpret is doing.
(The choice on how to construct the baseline is not well-agreed in the community, so they might be following another strategy)


Exhaustiveness checks on pattern matching

This should be a compile-time error:
let x = match None
    case (Some x) = x

console.log x

The match is partial.
Currently the above code outputs undefined.

In lieu of exhaustiveness checking, I'd add a runtime exception as the final case in all matches.  So the Roy
data Maybe a = Nothing | Just a
let x = match Nothing
    case (Just x) = x

console.log x

where it currently compiles to
var Nothing = function(){};
var Just = function(a){this._0 = a};
var x = (function() {
    if(new Nothing() instanceof Just) {
        var x = new Nothing()._0;
        return x;
    }
})();
console.log(x)

should instead compile to
var Nothing = function(){};
var Just = function(a){this._0 = a};
var x = (function() {
    if(new Nothing() instanceof Just) {
        var x = new Nothing()._0;
        return x;
    } else {
      throw("/tmp/foo.roy:(line,col)-(line,col): Non-exhaustive patterns in case");
    }
})();
console.log(x)


Exception during submission

@ThorpeJosh @siewyeng @lamchuntat

cmd : tp-timesheet -s today -n

@changukshin This should be resolved now. Can you please confirm on lastest version and reopen if it is still occuring?

Confirm. Working as expected.


What is the relationship between samples_per_gpu and workers_per_gpu and sample_ratio？

I can only use the default
sample_ ratio=[1, 4]
samples_ per_ gpu=4
workers_ per_ gpu=4

But to increase the batchsize a bit,
samples_ per_ gpu=8,
when the program cannot run and a len() error will occur. Can you tell me the relationship between these three and how labeled and unlabeled data is sampled in the project? Thank you very much

There are three distinct concepts to understand:

sample_ratio=[1, 4] indicates the ratio of labeled to unlabeled samples within a single GPU. For instance, sample_ratio=[1, 4] means that there is 1 labeled and 4 unlabeled samples on each GPU.
samples_per_gpu=5 refers to the total number of samples per GPU, regardless of whether they are labeled or unlabeled. In fact, sum(sample_ratio) == samples_per_gpu.
workers_per_gpu=5 determines the number of threads that will be used to load the data. The optimal number of workers per GPU depends on your server setup. By default, we set workers_per_gpu equal to samples_per_gpu, but you can reduce this value if your server has limited CPU resources. If necessary, you can set it to 1 or 0.

https://github.com/Adamdad/ConsistentTeacher/blob/1fa64775d93976d9b4ceffa4a2ee7d10a5c50c29/configs/consistent-teacher/consistent_teacher_r50_fpn_coco_180k_10p.py#L256-L296

Hello, I have been reproducing this paper recently and have encountered the issue you mentioned in another post. Loss=0. I trained on the Coco dataset, and before 10000, the unsup-loss and loss were both 0, but after 10000 the loss is noemal, but soon the loss became 0 again, and unsup_gmm_thr also became 0. Can we discuss this issue with you.

Hello, I have been reproducing this paper recently and have encountered the issue you mentioned in another post. Loss=0. I trained on the Coco dataset, and before 10000, the unsup-loss and loss were both 0, but after 10000 the loss is noemal, but soon the loss became 0 again, and unsup_gmm_thr also became 0. Can we discuss this issue with you.

请问您的该问题解决了吗，我也遇到这个问题

Hello @huangnana1  and @X-KL , how many GPUs are you using and did you change any values in the configs?

hello, thanks for your reply, reference your previous reply in other issues,  the problem I have solved. thank you very much.
---Original---
From: "Xingyi @.>
Date: Wed, Oct 9, 2024 14:21 PM
To: @.>;
Cc<EMAIL_ADDRESS>Subject: Re: [Adamdad/ConsistentTeacher] What is the relationship between samples_per_gpu and workers_per_gpu and sample_ratio？ (Issue #19)
Hello @huangnana1  and @X-KL , how many GPUs are you using and did you change any values in the configs?
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you were mentioned.Message ID: @.***>


Added logo the login component

What does this PR change?

It doesn't changes any functionality of the app. Trying to do my first contributions. Saw a issue popping "enhancement" which stated to add a large paralus logo to the main login page. So I decided to do it by using simple jsx and tailwind. The size of the iamge that loads is in 4k, checked in other throttle modes. Works fine.

Does the PR depend on any other PRs or Issues? If yes, please list them.

https://github.com/paralus/dashboard/issues/110

Checklist
I confirm, that I have...

[x] Read and followed the contributing guide in CONTRIBUTING.md
[ ] Added tests for this PR
[x] Formatted the code using npm run format (if applicable)
[ ] Updated documentation on the Paralus docs site (if applicable)
[x] Updated CHANGELOG.md

Hello @meain, I'll take a look of what you've mentioned tomorrow. I'll add the changes you mentioned.

may I know the url and query that makes the forgot password page working? And also the file where the forgot password components are exported as I cannot find it in Auth.js. So, I can take a look on the UI change you mentioned on the forgot password page

https://github.com/RafayLabs/rcloud-console-ui/blob/main/src/containers/Auth/KratosSettings.js is where you will find the password reset page which is loaded when you navigate to /ksettings. You can view this in the application when you have to reset your password (you will have to go through this when you create a new user).

lgtm, yeah let's change the text from Next to Login.

check this out

check this out

Let's fix the type-case also in this since we are working on this. The 'P' in both passwords placeholder text needs to be in caps i.e. "Password"

check this out

Let's fix the type-case also in this since we are working on this. The 'P' in both passwords placeholder text needs to be in caps i.e. "Password"

Hey @plagzee , looks like when this gets built and served via docker the paths are a bit different. The current images that we have in public are only used the manifest, rest of the images are coming from src/assets. Could you change use of images to something like below. You'll also have to move the images to the proper dir. Maybe rename the alt to "Paralus logo" as well I guess.
diff --git a/src/containers/Auth/KratosSettings.js b/src/containers/Auth/KratosSettings.js
index c6fe1f2..af8a801 100644
--- a/src/containers/Auth/KratosSettings.js
+++ b/src/containers/Auth/KratosSettings.js
@@ -8,6 +8,7 @@ import { useQuery } from "utils/helpers";
 import PageLayout from "./Login/components/PageLayout";
 import { createTheme, MuiThemeProvider } from "@material-ui/core/styles";
 import tealTheme from "../themes/tealTheme";
+import paralusLogo from "assets/images/logolarge.png"

 const KratosSettings = (props) => {
   const { query } = useQuery();
@@ -102,7 +103,7 @@ const KratosSettings = (props) => {
             autoComplete="off"
             onSubmit={handleChangePassword}
           >
-
+

diff --git a/src/containers/Auth/Login/index.js b/src/containers/Auth/Login/index.js
index 6c2ab88..8fc25f8 100644
--- a/src/containers/Auth/Login/index.js
+++ b/src/containers/Auth/Login/index.js
@@ -28,6 +28,7 @@ import { Redirect, withRouter } from "react-router-dom";
 import PageLayout from "./components/PageLayout";
 import ChangePasswordForm from "./components/ChangePasswordForm";
 import { newKratosSdk } from "actions/Auth";
+import paralusLogo from "assets/images/logolarge.png"

 const SESSION_TIMEOUT_MILLISECONDS = 12 * 60 * 60 * 1000 + 1 * 60 * 1000; // 12hrs + 1min
 class Login extends Component {
@@ -251,7 +252,7 @@ class Login extends Component {
       <>
         <>

@meain done


Error running Haste-boot on GHCJS El Capitan.

I installed the binary from the site.
Vs-MacBook:tests v$ ghc --version
The Glorious Glasgow Haskell Compilation System, version 7.10.2
Vs-MacBook:tests v$ haste-boot
Downloading haste-cabal from GitHub
Downloading Google Closure compiler...
Downloading base libs from GitHub
Reading package info from "libraries/rts.pkg" ... done.
Warning: The package list for 'hackage.haskell.org' is 46 days old.
Run 'cabal update' to get the latest list of available packages.
Warning: Topdown solver is deprecated
Resolving dependencies...
Configuring ghc-prim-<IP_ADDRESS>...
Building ghc-prim-<IP_ADDRESS>...
Installed ghc-prim-<IP_ADDRESS>
Reading package info from "ghc-prim-<IP_ADDRESS>-osx.conf" ... done.
ghc-prim-<IP_ADDRESS>: Warning: library-dirs: /Users/v/.haste/x86_64-darwin-haste-0.5.3-ghc-7.10.2/ghc-prim-<IP_ADDRESS> doesn't exist or isn't a directory
ghc-prim-<IP_ADDRESS>: import-dirs: /Users/v/.haste/x86_64-darwin-haste-0.5.3-ghc-7.10.2/ghc-prim-<IP_ADDRESS> doesn't exist or isn't a directory (use --force to override)
ghc-prim-<IP_ADDRESS>: cannot find any of ["GHC/CString.hi","GHC/CString.p_hi","GHC/CString.dyn_hi"] (use --force to override)
ghc-prim-<IP_ADDRESS>: cannot find any of ["GHC/Classes.hi","GHC/Classes.p_hi","GHC/Classes.dyn_hi"] (use --force to override)
ghc-prim-<IP_ADDRESS>: cannot find any of ["GHC/Debug.hi","GHC/Debug.p_hi","GHC/Debug.dyn_hi"] (use --force to override)
ghc-prim-<IP_ADDRESS>: cannot find any of ["GHC/IntWord64.hi","GHC/IntWord64.p_hi","GHC/IntWord64.dyn_hi"] (use --force to override)
ghc-prim-<IP_ADDRESS>: cannot find any of ["GHC/Magic.hi","GHC/Magic.p_hi","GHC/Magic.dyn_hi"] (use --force to override)
ghc-prim-<IP_ADDRESS>: cannot find any of ["GHC/PrimopWrappers.hi","GHC/PrimopWrappers.p_hi","GHC/PrimopWrappers.dyn_hi"] (use --force to override)
ghc-prim-<IP_ADDRESS>: cannot find any of ["GHC/Tuple.hi","GHC/Tuple.p_hi","GHC/Tuple.dyn_hi"] (use --force to override)
ghc-prim-<IP_ADDRESS>: cannot find any of ["GHC/Types.hi","GHC/Types.p_hi","GHC/Types.dyn_hi"] (use --force to override)
Command '/Users/v/.cabal/bin/haste-pkg' failed with error  code 1

Running haste-boot should never be necessary with the binaries; in fact, the command should not even be present. What is the output of which haste-boot?

I was using a version installed with cabal.

@valderman I'm running into the same error and found the cause for the missing ghc-prim if you're interested:
Internally when running "haste-boot" some files are downloaded into a folder, which should not be created, because its a duplicate then. So this seems to be a simple path-problem, i'll attach a screenshot from Finder, which shows it.

/Users/v/.haste/x86_64-darwin-haste-0.5.4-ghc-7.10.2/ghc-prim-<IP_ADDRESS> obviously doesn't exist (since its inside another x86_64-osx-haste-0.5.4-ghc-7.10.2-folder and the whole thing fails...
So "Running haste-boot should never be necessary with the binaries;" means just cabal install haste-compiler is enough and hastec works then?
Thanks in advance!

No, if you install from source (i.e. using cabal, either from GitHub or from Hackage) you always need haste-boot. However, the binaries available from haste-lang.org are pre-booted and should work right out of the box, no booting required.
Regarding the error, it seems like yet another case of GHC behaving completely unpredictably WRT paths on OSX. I don't have a mac available so I'm not sure when I'll be able to sort this out, unfortunately. Patches to fix this and other OS-specific issues are particularly welcome.

Thanks for your quick reply! @valderman
I didn't try to use hastec right after binary install, since the documentation on the main github-page says:
Or, you can install the latest stable version from Hackage:
$ cabal install haste-compiler
$ haste-boot
I tried to compile from source after i got into the haste-boot errors, so this might be the cause for the necessary booting then.
However, i just successfully booted it a couple mins ago by copying and inserting the ghc-prim-<IP_ADDRESS>-folder to the right place at the right time during haste-boot (dirty workaround, but i needed a working version asap). For me this looks like a simple path-problems in one of the scripts, no?!
I'll try to translate some Haskell code to JavaScript for a Webapp now, thanks again! :smiley:


Almost complete rewrite

Rewrote the entire application for better *NIX support(displaying file permissions)
Needs testing on Windows
Should have slightly better performance
Unfortunately, new tests will need to be written as I got caught up in the process and rewrote everything 💀

Awesome man! Sorry for the late response, I will test here and soon give a feedback

Yea, this relies on a lot of conditional compilation that I have not tested to work on Windows. Hopefully it works, if it doesn't, I'll try to fix it!


Pub/Sub component with Session enabled Azure Service Bus Topics do not deliver messages in sequence

Expected Behavior
When sessions are enabled, a Pub/Sub component subscription should receive messages in sequential order.

Actual Behavior
Messages are being received out of sequence, with the next message arriving before the previous one has been received.

Steps to Reproduce the Problem

Create a DAPR application.
Configure a pub/sub component to connect to an Azure Service Bus Topic.

[ApiController]
public class PartiesCommandsController(
    ConventionNamingCommandProcessor commandProcessor,
    IHostEnvironment hostEnvironment,
    ILogger<PartiesCommandsController> logger) : CommandSubmissionController(commandProcessor, hostEnvironment, logger)
{
    /// <summary>
    /// Handle external systems commands as an asynchronous operation.
    /// </summary>
    /// <param name="commandState">State of the command.</param>
    /// <returns>A Task<ActionResult> representing the asynchronous operation.</returns>
    [CustomerCommandsBusTopic]
    [TopicMetadata("requireSessions", "true")]
    [TopicMetadata("sessionIdleTimeoutInSec ", "2")]
    [TopicMetadata("maxConcurrentSessions", "8")]
    [HttpPost("/handle-customer-commands")]
    public async Task<ActionResult> SubmitCustomerCommandsAsync(CommandState commandState)
        => await HandleCommandAsync(
                commandState,
                Customer.GetAggregateName(),
                CancellationToken.None)
            .ConfigureAwait(false);
}

I observe the below setup, but it appears to be globally applied to the topic. This configuration might not support concurrent session handling if it is set to 1.

maxActiveMessages : Defines the maximum number of messages to be processing or in the buffer at once. This should be at least as big as the maximum concurrent handlers.
maxConcurrentHandlers : Defines the maximum number of concurrent message handlers.

Release Note

RELEASE NOTE: Fix session enabled topics should send messages in sequence

By design messages are received for the given session first, unless within sessionIdleTimeoutInSec window there are no messages available. At which point the next session will be tried.
Up to maxConcurrentSessions sessions will be tried in parallel within a given sidecar.
If you have multiple sidecars this of course further adds to the parallelization.
Just like in the image you provided, session 1, 2 and 3 would be processessing / receiving messages in parallel - assuming you have at least 3 maxConcurrentSessions. There is no guarantee made by Dapr for which session will start first. And there is no such thing as perfect timing between these sessions - each session is a different goroutine and the timing of those can vary. But for each goroutine, as long as the session as new messages sessionIdleTimeoutInSec apart, then the messages within that session are received in order.
Absolute ordering across multiple sessions is impossible and not a support use case.

You may need to reduce the number of sidecars to avoid processing the same session multiple times in parallel.

@berndverst Thank you for your responses. My primary concern is not the sequence in which the DAPR sidecar processes the sessions. When performing incremental updates for customers, it's crucial to ensure that all messages related to a specific customer (using the customer ID as the session ID) are processed sequentially. If a systemic error occurs in my application, there might be several messages with a retry count greater than zero. However, this would occur only once per session. As shown in the screenshot, multiple messages from the same session have a retry count exceeding zero. The application operates in an Azure Container App with a single sidecar (Scale 1-1).

@jpiquot thanks for raising the issue, happy to try and help.
Based on what you've shown can you just clarify that your application is always erroring and never returning a successful response?
As I can see the messages in your subscription in the message browser I am assuming there is no dead letter setup - is this correct?
Each message appears to be delivered 9 times. This does not mean it has been sent to your app 9 times. Rather it means we have taken the message lock and consumed it 9 times. Dapr resiliency may increase the time we hold the lock whilst retrying the call to the app which could expire and increase the delivery count. However, this should not allow a consumer to take a lock on next message in the session sequence. Do you have any idea why we are consistently seeing 9 delivery count?

@jjcollinge thanks for your answer.
Messages at the top of the queue are retried up to ten times, in line with the queue's settings for a maximum of 10 retries, before being moved to the dead letter queue, which is currently empty since I have manually cleared it.

Thanks for the context.
So just to clarify my understanding. The subscriber is intentionally returning an error on each message that is delivered. This causes the message delivery to fail up to the maximum of 10 times. Service Bus then removes that message from the subscription and moves it to the dead letter queue. The subscriber then takes the next message in the session and continues the cycle. From what I can see in your message explorer the sequence numbers looks to be in order. The reason for many messages having a delivery attempt is due to the message being dropped after 10 attempts and the consumer moving to the next one. Which bit of the behaviour are you not expecting?

In the list, multiple messages within the same session have an incremented retry count (none are dropped yet). This contradicts the idea of a FIFO message delivery. If message one has retry counts, message two should not have any retries.
The session pattern's primary goal in pub/sub systems is to enable parallel processing of message collections while ensuring that messages within a single collection are delivered in a serialized, FIFO sequence. This ensures that each message is processed in the order it was sent, maintaining the integrity of the message sequence (important for DDD aggregate events).
Moreover, as evident in the Azure Service Bus message list, more than one message has been delivered within the same session, each with a retry count. If the message delivery were functioning correctly as per the FIFO principle, we wouldn't observe multiple messages with incremented retry counts in the same session.
This situation occurs even with a single active sidecar. If there were multiple pods, I would expect the sessions to be distributed across them, but still, each message in a session should be delivered in sequence to the same pod.

Ok sorry I am away from my laptop for the holidays so only viewing your image on my phone. I mistakenly thought the messages were being dead lettered in order but now I understand what you mean. I'll have to try and reproduce this when I'm back in the new year.

Thanks for your reply. Yes sessions are required in the subscription.

@jpiquot I'm struggling to repro this in testing... I've added a new certification test that publishes 100 messages to 3 sessions concurrently and ensures that with max concurrent sessions set to 8 on 2 subscribing replicas we receive each session in order which is working as expected. Looking back at what you've explained... your app deliberately returns an error when it receives a message. You said you see multiple messages within the session with a delivery count greater than 0 but less than 9 at the same time, correct? Are you able to validate that your app is actually receiving requests out of order just to rule out that this is some eventual consistency in the Service Bus explorer. I'll continue trying to reproduce this with a test in the meantime.

@jjcollinge thanks for your help. The issue may indeed stem from a specific failure condition in our application. Initially, I suspected the root cause to be related to message sequencing. Observing multiple messages in the same session with a retry count seemed to confirm this.
We identified a problem in the our command handler, which was issuing some events with an invalid session ID (excessively long), thereby creating an error condition.
To monitor this, I need to implement additional state management in the command processor. This will involve storing the message sequence and ensuring that we never receive a message with a sequence number lower than its predecessor. This change will aid in identifying any issues related to message order.


BUG: Flipkart Icon is not showing

Is there an existing issue for this?

[X] I have searched the existing issues

What happened?

In this image you can clearly see that Flipkart icon is not showing.
Record

[X] I agree to follow this project's Code of Conduct
[X] I'm a GSSOC'24 contributor
[X] I want to work on this issue

@arghadipmanna101 Please assign this issue to me

@arghadipmanna101 Please assign this issue to me

Okk..! I'll check the problem.

Made exact same logo using Adobe Photoshop.
KIndly merge my PR.

I would like to contribute to this issue please consider me ...

Already solved


Cannot play a redirected episode

Describe the bug
It seems that Anytime cannot play op3 redirected episodes. Maybe an issue with the enclosure URL?
To Reproduce
Steps to reproduce the behavior:

Go to 'Curry and The Keeper' podcast
Click on the play button of episode 22
Episode isn't playing

Expected behavior
Should play like other episodes
Screenshots
If applicable, add screenshots to help explain your problem.

Smartphone

OnePlus 8T
Android 11

Additional context
I was told this specific episode is using a redirected enclosure URL.

I found the problem, there is a space on the URL: <enclosure url="https://op3.dev/e/mp3s.nashownotes.com/CATK-22-2022-09-28-Final.mp3 " type="audio/mpeg" length="59004805"/> http://feed.nashownotes.com/catkrss.xml
I'll trim the URL before trying to download it to fix this bug.


fix: support maxSize set null and deal with ENOENT error issue when rotated in pm2 cluster mode

1、支持maxSize设置为null，即不开启按大小轮转
2、防止pm2开启多实例时，当日志轮转并被压缩时，原日志文件被删除后，导致部分实例stat原日志文件抛出异常
3、按标准prettier部分代码格式

Codecov Report

Merging #68 (c7df4bc) into main (d2977ce) will decrease coverage by 0.15%.
The diff coverage is 83.33%.

@@            Coverage Diff             @@
##             main      #68      +/-   ##
==========================================
- Coverage   84.79%   84.64%   -0.16%
==========================================
  Files          13       13
  Lines         921      925       +4
  Branches      276      271       -5
==========================================
+ Hits          781      783       +2
- Misses        128      130       +2
  Partials       12       12

Impacted Files
Coverage Δ

src/transport/fileStreamRotator.ts
83.93% <80.00%> (-0.07%)
:arrow_down:

src/logger/logger.ts
95.16% <87.50%> (-0.51%)
:arrow_down:

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


Setup a 2nd prod environment

Overview
We want to setup a 2nd prod environment in SE Asia and add that to the front door

@codyborn, spoke to @eruizgar91 about this and he said that he would need you to setup the resources in Azure first and then he can take it from there. Let us know if that's ok or you want to approach things differently.

@bowd that works for me. I'll setup the infra and fund the new relayers today.


Windows installer links appear to be broken

Clicking on "Download the MSI Installer" (https://azurecliprod.azureedge.net/msi/azure-cli-2.0.41.msi) is resulting in a blob not found error:
This XML file does not appear to have any style information associated with it. The document tree is shown below.
<Error>
BlobNotFound
<Message>
The specified blob does not exist. RequestId:000ce74d-001e-0006-620f-134d9c000000 Time:2018-07-03T20:47:14.2773601Z
</Message>
</Error>

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 8246b90a-efb1-03ce-f642-d3eb2441fc30
Version Independent ID: 2fdd20ef-7bd3-57bc-7957-93f4bb5377ce
Content: Install the Azure CLI for Windows
Content Source: docs-ref-conceptual/install-azure-cli-windows.md
Product: azure
GitHub Login: @sptramer
Microsoft Alias: sttramer

This is a duplicate of #917. The 2.0.41 release is supposed to be out this afternoon, and the redirect the link points to was updated before the MSI was available.

Please retry.

Thank you!  As a quick note:  I did have to clear the cache on the page that presented me the error before I could continue with the download (Click download link > End up on same blob not found error > Ctrl + F5 to clear the cache and reload the page for a successful download).


Added Ruby 3.2 to workflow.

Description
This PR is for adding Ruby 3.2 to your workflow.
How about this PR?
Test
I have confirmed that the Ruby test succeeds in my repository.
https://github.com/madogiwa0124/omniauth/actions/runs/4094164006

Part of this was fixed in #1113


Pressing shuffle button on other playlist then "All" crashes

it happens on the new update 01.5

Oh man, I thought I considered the versioning of the Playlist file when I included the per-playlist shuffle stuff... It seems I didn't
and it is crashing because it expects the next version.
I just fixed it and will update the release again in 1-2minutes.
So I hope this will not happen after you get the newest release again.
If so, you can open this issue again.


Document the other tuple being used for ClientIP, ClientIP & Protocol distribution modes

Explain in this article what the other unspoken tuple is.  As a layman, I would assume that it's Destination port, or Destination IP, but this is unclear.
FTA:
None (hash-based) - Specifies that successive requests from the same client may be handled by any virtual machine.
Client IP (source IP affinity two-tuple) - Specifies that successive requests from the same client IP address will be handled by the same virtual machine.
Client IP and protocol (source IP affinity three-tuple) - Specifies that successive requests from the same client IP address and protocol combination will be handled by the same virtual machine.

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 4df3fe8d-86bf-23bf-3f5b-b9cce432412e
Version Independent ID: 42eb0bc5-e329-8d92-c850-b0baed07ec98
Content: Configure Azure Load Balancer distribution mode - Azure Load Balancer
Content Source: articles/load-balancer/load-balancer-distribution-mode.md
Service: load-balancer
GitHub Login: @mbender-ms
Microsoft Alias: mbender

Hi @caroysMSFT, thanks for your feedback. We are implementing changes to the document with your feedback.
#please-close


ngAnimate is not working if ng-app is not defined

Angular do not add ngAnimate enter, leave classes if ng-app="app" is not defined. Does someone has same issue?

You mean if you bootstrap the app manually? Which version are you using?


Ant mkdir task is referenced out-of-nowhere in "Creating directories" chapter

Issue type
Missing information
Problem description
The documentation talks about Java API to create files, but then suddenly mentions an Ant task.
Context (optional)
No response
Page with the problem
https://docs.gradle.org/current/userguide/working_with_files.html#using_file_mkdirs_and_files_createdirectories

This is a valid documentation issue.


AddressSanitizer: allocator is out of memory

version: 2.4 12.0
./bin/iconvert crash_000122 1.png
=================================================================
==121851==ERROR: AddressSanitizer: allocator is out of memory trying to allocate 0x3b7f84804 bytes
    #0 0x4c9e6d in operator new[](unsigned long) (/root/oiio/build/bin/iconvert+0x4c9e6d)
    #1 0x7f22ca2b4b9d in OpenImageIO_v2_4::ImageOutput::copy_image(OpenImageIO_v2_4::ImageInput*) /root/oiio/src/libOpenImageIO/imageoutput.cpp:587:36
    #2 0x4d169b in convert_file(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) /root/oiio/src/iconvert/iconvert.cpp:449:27
    #3 0x4cd218 in main /root/oiio/src/iconvert/iconvert.cpp:523:14
    #4 0x7f22c87f2082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/../csu/libc-start.c:308:16

==121851==HINT: if you don't care about these errors you may set allocator_may_return_null=1
SUMMARY: AddressSanitizer: out-of-memory (/root/oiio/build/bin/iconvert+0x4c9e6d) in operator new[](unsigned long)
==121851==ABORTING

poc:
crash_000122.zip

It's out of memory when trying to allocate 16GB. Could it just be that your machine didn't have enough free RAM?

But I think the point is that it only looks like a 16GB image because it's a corrupted file with an absurd resolution.
There is an int limits:imagesize_MB attribute added recently -- a sanity check on image size, where images that purport to be larger than this are presumed to be corrupted or malicious. For apps that actually expect images larger than this, you can, of course set it as large as you want. The default is 32GB.
Question 1:
Should it be a lower default? It was hard to know the tradeoff between catching allocation errors from corrupt files, versus "false negative" cases of rejecting legitimately huge images. But maybe the 32GB default (for a single image, remember) is still drastically too big considering how commonly people will run on VMs with lower memory limits than this.  For comparison, a 32k x 32k RGBA uint16 image is 8GB uncompressed.)
Question 2:
This limit attribute is informational only, it's up to each format reader to check the allocation size against this limit. PNG currently does not! So that should be fixed -- but note that even if we fixed it today, it would happily let the allocation happen if the default is 32GB.
Crazy idea: What if instead of defaulting this limit to a flat 32GB, we set it to min(32GB, physical_memory/2)? The advantage is, when running (or doing CI, or fuzz-testing) on a low-memory machine, it still will catch what are likely to be errors. The disadvantage is, the default of the attribute might depend on which machine you are running on. But then again, it's still enormous for a single image, and sophisticated apps that might really deal with giga-images can always proactively set the attribute to a size they are comfortable with.

Sanity check so that there's a clean early out is nice, especially when the machine does have TBs of RAM but we don't want to wait hours for it to try to process a corrupt image.
But I don't think limits:imagesize_MB will solve the fundamental issue raised here, which is that we need to check the result of all of our thousands of new/malloc calls to see if they succeeded and then properly handle these failures. That's a lot of work and clutters the code, so I don't know if it's worth it. The reason I think this won't help with the fundamental issue is that whatever value you pick, it's still possible to fail. Maybe the user has 32GB of RAM, so physical_memory/2 returns 16GB and this 16GB allocation request is allowed to happen. But let's say free memory was only 32MB at this stage so of course it failed.
Maybe we need to be strategic and find the few places where we are likely to allocate GBs of data (anywhere that allocates all the pixels?) and check if the allocation passed?

May be relevant: https://stackoverflow.com/questions/18684951/how-and-why-an-allocation-memory-can-fail
I think the intent of limits:imagesize_MB is actually not to catch or recover from bad allocations (which is both difficult to catch everywhere in the code base, and as noted in the link above, may not work the way you think on modern systems).
Rather, it's intending to catch corrupted or malicious files that have implausible resolutions. Like you said, even if a fishy allocation "succeeds", it may be so large that it's effectively a DOS attack. Ideally, every reader should do this check. In practice, I implemented it for TIFF (adding limits:imagesize_MB in the process) but only fixed that one error I was facing at the time (very similar to this one -- the result of fuzz testing TIFF files) and never quite got around to revisiting all the other readers.

Side note: the POC file here is 21 bytes, so this is not in fact a large image, it is a corrupted/malicious header masquerading an image file, and the resolution it describes is implausibly large.

Yeah, I've always interpreted that limit as just a safety net for "zip bombs" (a small file, 21 bytes here, expanding out to several gb of memory): https://en.wikipedia.org/wiki/Zip_bomb
Writing software that properly handles new returning null or bad_alloc is... rarely ever done, attempted, or gotten correct.  In this particular case I'd be more concerned if the operation that triggered the memory allocation eventually fails gracefully when it realizes there's not really 16gb of data to read :)

On my laptop, with plenty of memory for a 16GB allocation, it still fails very quickly with:
$ iconvert crash_000122 1.png
iconvert ERROR copying "crash_000122" to "1.png" :
	Read error: hit end of file

So at least it does detect in other ways that the file is bad before getting into too much trouble... unless the allocation fails.
It's hard to tell from this error message exactly which reader caused the problem (because there is no file extension and it's not valid anyway, it will try them all). Turns out the bit pattern of the file looks like a Targa file, and that's the reader that hits the EOF while reading the pixels.

That error message I get is not a crash, by the way. It's a graceful exit with error message.
The crash only happens if we try the unacceptable allocation.

Notice how the stack trace from address sanitizer says it failed INSIDE operator new. Thus, the application (a) never got back a null, (b) nor did it fail to catch a bad_alloc exception. It just dies inside new, so there is nothing we can do in our code to handle the problem if the allocation fails, because control never returns to us. We have to detect the allocation size as implausible beforehand and just say no.
Back to my suggestions:
I feel like lowering the default is an independent question -- it's valid to ask if 32GB is too high a default, but at the same time, if we lowered to 16GB, there certainly exists a VM somewhere where even that can fail. (We have to ask ourselves, though, if we care.)
I still kinda like my suggestion about lowering the default to be at most some fraction of physical memory size to try to catch it regardless of how constrained a system it is.

I have a patch to suggest, coming shortly.

Proposed fix: https://github.com/OpenImageIO/oiio/pull/3954

That error message is a graceful exit when running under address sanitizer because address sanitizer has instrumented the code to look for nullptr returned from new and added code to gracefully exit. The real OIIO code would likely have crashed once it tried to access this null pointer unless it already is handling this case.
I still think there's nothing we can really do for this particular issue short of adding null checks after the allocation, which sometimes will help and other times won't for the reasons lg linked to already. Machines with enough memory, like lg's will be able to do the allocation. Machines without enough memory will crash. Tweaking limits:imagesize_MB can maybe help in some cases but it would be trivial to create a new file that still crashes in certain situations. What's more, I'm worried about users with say 4GB of RAM that will now no longer be able to operate on legitimately sized textures even though they have the SWAP space to do it.

As it turns out, it wouldn't have crashed outside the sanitizer (I think???), because I found the exact site of the new call that fails, and it is in fact surrounded by a try/catch block to detect a failed allocation.
You are right, the physical_memory/2 trick is a little hokey and can backfire on people with 4GB of RAM as you say. But how often is that really the case, the combination of huge image, tiny memory, and user/app that doesn't proactively raise the attribute to reflect the size of image they expect?
If you or others don't think that changing the default based on the physical memory is a good idea, I can back out of that aspect of the PR.
But I'm not sure what is the best thing to do in this case.
I hear what you're saying about the crash in new being an artifact of address sanitizer. The dilemma, though, is that there are people out there running fuzz tests against the software with the sanitizer, and will report these crashes here (or file CVE's) when these failures occur, unaware that they are seeing a crash that can only happen for the address sanitizer. Yet each one will still require us to track it down and verify that without the sanitizer in place, we correctly catch the failed allocation.
So I still want a way, even in the sanitizer case, to find a likely-to-be-mistaken image size ahead of the allocation in order to stem the tide of these pseudo-bugs being reported.

I can imagine a user app is oiiotool or maketx. Do these tools proactively raise the attribute? If it requires a special flag, that's extra complexity and chances are a user that doesn't have huge amounts of RAM probably is not a power user and so will have an extra hard time knowing how to pass the appropriate flags to overcome this. Doubly so if these tools are being wrapped by a script/GUI.
As for user/machine, it's not that rare for non-professional users, such as students, to have a modest machine.  8GB of RAM until recently was the default for most mac laptops. So half of that is 4GB. I'll admit 4GB is still a pretty big and rare image, so perhaps the majority of users would still be fine with this limitation. So I won't fight against the divide by 2 proposal if you think it's needed.
For the fuzz tests, it would not surprise me if eventually we get people running them on VMs that have a few GBs of RAM, no swap, and have much less than half of their RAM still free. They'll report issues which none of us will be able to repro on our production hardware and we'll spend extra time trying to figure out what is going on. I do like your suggestion in the PR of checking if address sanitizer is running. I suggest though getting very aggressive. Don't divide by 2. Divide by 8. You can do this with  __has_feature(address_sanitizer) as explained in https://clang.llvm.org/docs/AddressSanitizer.html#conditional-compilation-with-has-feature-address-sanitizer.

I was thinking of "don't divide by 2" also, but by just using the physical memory itself. The rationale is that it gives more tolerance for users with small mem computers, and it still signifies an important boundary -- if they try to read images larger than this into memory, at the very least they will see swapping and a huge reduction in performance.
The error message itself SAYS why it's rejecting the file and the name of the attribute to set to make the error go away if they really do intend to have files that large.
I'd like to think (but could obviously be wrong) that the combo of { small machine, not a power user } might be fairly unlikely to have multi-GB images that aren't mistakes. Remember, for reference, that a 16k x 16k x RGBA x half image is still only 2 GB.
The GitHub runners have 8GB VMs, BTW.
So I think the choices we have before us are:

Fixed limit, but lower than the current default of 32 GB. I fear that on small VMs or old laptops, they may still fail these fuzz tests unless we lower so far that we might have users with legit big images that trigger the warning.

Fixed limit, but go much lower (maybe 4GB?) when address sanitizer is detected. This will gracefully error/fail for a lot of these corrupted files when being fuzz tested, but we are taking it on faith that when not using the sanitizer, will will catch the corrupt files in another way.

Limit of min(fixed_default, fraction * physical_memory), so low memory situations (not just sanitizer) catches both sure-to-fail allocations and also will-succeed-but-surely-will-swap bad performance cases. (The value of fraction is up for debate, but I'd be ok with either 0.5 or 1.0.) Remember that the error message itself says what attribute to alter if this is a false positive.

Like (3), but only cut down the value with a physical memory limit in the sanitizer case.

Unknown other solution that hasn't occurred to us.

Allocations that "succeed" but lead to terrible performance aren't really desired anyway. I still think there's merit to the default size for the sanity check size to be somehow tied to the physical memory, but maybe 0.5 was too aggressive?

Option 2 might prevent useful fuzzing tests where we want to detect bugs when exceeding 32-bit counters. If the machine has enough memory, they should be allowed to test that. So option 3 or 4 are the ones I vote for since it avoids the bulk of the out of memory cases while still letting those with lots of RAM to use OIIO without noticeable limitations.
For 4, I would use the 0.5 multiplier or smaller. For 3 (real users) I would maybe suggest 1.0 so that we're a little bit more lenient.

I'm currently leaning toward 3, but raising the fraction to 1, so that the only cases where we might have a false positive for small-memory users are ones where they will end up swapping anyway. We can always revisit if this is causing problems for people.

I'm making that change and will update the PR shortly.
Just to reiterate a very subtle point: Just because a full uncompressed image is larger than available memory doesn't necessarily mean that it couldn't be processed in certain ways by OIIO (consider the case of carefully reading, processing, and writing one scanline at a time, so the working set can be tiny). This limit is not "how big an image is too big to process," but rather it means "how big is implausible to be a 'real' image rather than a corrupted one." There is no correct answer to that question, it's a heuristic that we are using to catch many corrupt images while having very few false positives.  And we're assuming that users are unlikely to be encountering legit images that exceed the physical memory of the machine they have chosen for the task, unless they are aware that they are doing something unusual and raising the limit.


VecSkipError fails on tagged enum

I have the following example;
struct RGB { r: u8, g: u8, b: u8 }

enum Pixel {
    RGB(RGB),
    Grey(u8),
}

struct Picture { pixels: Vec<Pixel> }

impl<'de> serde::Deserialize for Picture {
    fn deserialize<D>(deser: D) -> Result<Self, D::Error> where D: serde::Deserializer<'de> {

        // this works
        let pixels: Vec<Pixel> = serde::Deserialize::deserialize(deser)?;

        // this fails with 'expected "string or map", found: "a sequence"'
        let pixels: Vec<Pixel> = serde_with::VecSkipError::<serde_with::Same>::deserialize_as(deser)?;

        Ok(Self { pixels })
    }
}

I have tracked the issue down to this match. At which point, the value of err reads 'expected "string or map", found "a sequence"'.
My tagged enum is getting serialized as a Seq. It appears that the inner content deserializer doesn't expect nested sequences? I'm not sure how to continue investigating, but deserializing straight to Vec works fine with normal Serde, I would expect VecSkipError to just drop in. I cannot use the derives because my types have other constraints.
Thanks!

VecSkipError is not a drop-in replacement as it suffers from https://github.com/serde-rs/serde/issues/1183
To continue after a deserialization error, you need to buffer the data, to ensure the Deserializer is always in a good state.
Can you please provide more details for your bug report? Without knowing how your serialized data looks and what data format you are using, there isn't anything actionable for me to do here.

Thanks for responding so quickly.
For some reason my struct RGB is being serialized as a Seq of 3 u8s. I am using normal serde on the serialize side, and deserialize_as is expecting a Map. Do I need to also use serialize_as in some way for round trip compatibility?
My serialize function;
fn serialize<S>(&self, ser: S) -> Result<S::Ok, S::Error> where S: serde::Serializer {
	self.pixels.serialize(ser)
}

I'm using RON, and my data appears as;
pixels:[(RGB((r:10,g:10,b:10))),(RGB((r:20,g:20,b:20)))]
I believe the issue is the inner parenthesis around the integers. I'm not completely clear on why deser thinks (r:10,g:10,b:10) is a Seq, as in RON this is struct notation.

RON is a complicated format. Its data format is more complicated than what can be modelled well with the serde API. This means it needs to have the context of the Rust types for proper deserialization. Any buffering will cause problems, for example https://github.com/ron-rs/ron/issues/500. Maybe the v0.9 design of RON will help a bit.


deps: Set Dictionaries 0.3 -> 0.3, 0.4

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
4 unchanged lines in 1 file lost coverage.
Overall coverage decreased (-9.1%) to 73.171%

Files with Coverage Reduction
New Missed Lines
%

src/levels.jl
4
56.0%

Totals

Change from base Build<PHONE_NUMBER>:
-9.1%

Covered Lines:
60

Relevant Lines:
82

💛  - Coveralls


Update copyright message

This PR updates the copyright message:
Copyright (c) 2021 Core Rule Set project. All rights reserved.
becomes:
Copyright (c) 2021-2022 Core Rule Set project. All rights reserved.

Thank you.
Would you mind updating the date in the LICENSE file as well in this PR? It's been a while it seems. :)

Good spot. LICENSE file copyright now brought into line with all the others.
Merging now.

Great. Thanks.


Pool_stat is empty, even though pool_stat sync configuration is enabled

OS
Your OS: Ubuntu 22.04
Versions
The db-sync version (eg cardano-db-sync --version): <IP_ADDRESS>
PostgreSQL version: 12
Problem Report
I'm using <IP_ADDRESS> version to sync onchain-data, and pool_stat sync configuration is enabled (please check my config)
"insert_options": {     "tx_cbor": "enable",     "tx_out": {       "value": "enable"     },     "ledger": "enable",     "shelley": {       "enable": true     },     "multi_asset": {       "enable": true     },     "metadata": {       "enable": true     },     "plutus": {       "enable": true     },     "governance": "enable",     "offchain_pool_data": "enable",     "pool_stat": "enable",     "json_type": "text"   },
But pool_stat table is empty. Please help me

There is an issue in last release, where you need to provide pool_stats instead of pool_stat. This will be fixed for the coming release pre-release https://github.com/IntersectMBO/cardano-db-sync/pull/1878


Document how to "reset" data to clean slate

It would be awesome to have a script or just docs detailing how we can reset to a clean slate. I mean to wipe out all data in whisper / graphite, and start from scratch without rebuilding the VM.

Uh, huhuh. https://github.com/obfuscurity/synthesize#removal

@obfuscurity I don't mean to remove the installation of all the packages and configuration, just to clear the existing data. I'm trying to just optimize the time of iteration for testing streaming data, clearing data, and then streaming in a new set.

I think it might be as simple as nuking the whisper directory, but not sure

Oh, well yeah, that's easy.
$ sudo rm -rf /opt/graphite/storage/whisper/*

Thanks!


Bring Travis CI configuration up-to-date with supported PHP releases

@chris-hamper Thanks for filing this PR, I understand where you're coming from and agree there's definitely some use for this!
That being said, I will have to reject this PR for now as it doesn't provide any substantial value for consumers of this package and also locks out a non-zero number of dependents of this package. This may change in the future of course, but I'm still seeing a noticeable number of installation on PHP versions that are not officially supported by Zend, but are still officially supported by a number of common vendors and distributions. I understand this is a controversial topic and do not wish to start a heated discussion about this decision. Thank you for your understanding.
Despite this, I would again like to thank you for your contribution, this is very much appreciated, even if I didn't merge it this time. Please don't see this as discouraging and keep it coming!


feat: allow definition of sql type

New feature
Add a new field to the following column strategies:

literal
fake_update

It allows user to define it's own SQL type. This is to override a limitation of types.
Use case
With the following database schema in PostgreSQL:
CREATE TABLE foo (
    id uuid PRIMARY KEY,
    sensitive_data uuid
);

If I write the following strategy file:
tables:
  foo:
    columns:
      sensitive_data: uuid4

We got the following error:
ERROR:  column "sensitive_data" is of type uuid but expression is of type character varying

Solution
To overcome this issue, we could fix the strategy file by writing this:
tables:
  foo:
    columns:
      sensitive_data:
        type: fake_update
        fake_type: uuid4
        sql_type: uuid # <--- new field

In the seed table, we will have the correct data type to update foo table.

I have a couple of things in the implementation to discuss:

I want to be sure of the use-case. Can you think of any other cases(like this one) where you'd want a value from faker that comes out as a string, but you store it differently in a database? UUID is an easy one, but for others i'm coming up a bit short. I'd like to put these in the documentation so everyone understands what the new feature is all about!

I am not really sure for the other use cases. I haven't tested it but maybe these types can collide:

json and jsonb
geometric data types
postgis data types

Also, as the type is chosen by pynonymizer, the user is not able to create BIGINT for example.
But that depends on the use cases and I am not able to find others apart from mine. 😅

Maybe we could modify the _get_column_subquery to use sql_type to cast the text field to something else?
Something along the lines of:
    elif column_strategy.strategy_type == UpdateColumnStrategyTypes.FAKE_UPDATE:
        return f"( SELECT {column_strategy.qualifier}::{column_strategy.sql_type} FROM {seed_table_name} WHERE \"updatetarget\"=\"updatetarget\" ORDER BY RANDOM() LIMIT 1)"

Yeah sure, I will update my PR then 😄

Also, as the type is chosen by pynonymizer, the user is not able to create BIGINT for example.

This is legit. What i've tried to do in the seed table is pick types that meet or exceed what faker/python gives us, and then worry about the update statement - I think in the case of bigint we could just make that the default and then have it cast "down".
In the case of complex types like geometric/json, whos equivalent would be something like a class or a dict... I'm not sure, in short. I don't want to add a big serialization concern to the anonymization part just to support them, when the literal type and this feature can get you most of the way there.
I've added on: pull_request to the workflow on master, btw, I'm not sure if that will take effect on next commit or what. I really need to see the tests pinging off since there have been a few things here . You might need to merge/rebase from upstream.

hey @babariviere, I think there might have been a misunderstanding regarding what i was asking for in the last review. I think that testing for the casting behaviour working is a good idea(!) I don't want to revert the tests you've added - I think that the fixtures being used specifically should have get_data_type return FakeDataType.STRING instead of the string 'UUID'.
Other than that It's looking great. I can't wait to merge it! ✨

@jerometwell Oh sorry ! I will put it back then ! 😄


Large 3D reconstructed world import question

Hi guys, our team attends to import large 3D reconstructed world (larger than 1 GB) and the texture file is not easy to be combined into one texture file. I have downloaded the gibson dataset and matterport dataset, but their sizes are all no more than 200 MB. So I wonder that whether you considered or tried to import large 3D world into iGibson? Thank you！

OK, I have imported a large obj file (more than 800M) successfully in static scene, but the FPS is relatively slow (3~4). And when I failed to add robot in it or integrate it with ROS with exception of segment fault (core dumped). So I suppose that iGibson cannot handle large 3D constructed world.

@fxia22 Any ideas?
I would suggest maybe modifying the texture_scale to 0.5 and see if there is any speedup?

I have tired reducing the texture_scale to 0.5, and the FPS rises from 3-4  to 4-5.

How large is the space itself? We have reconstructed large spaces and manage to represent it efficiently with a small 3d file. You can consider doing some mesh decimation to make it smaller, blender would be an excellent tool for that. tutorial

I suspect the problem is not texture size, but it is caused by obj file with too many polygons.

@fxia22 Thanks for your reply! The space itself is about 500-600 square meters. We have tried downsampling using Blender to reduce the .obj file size, but it will cause some empty space which may harm robot movement. The vedio you provide is really helpful and I assume making such adjustment is very time-consuming lol.


Adding navbar component to app

Nice, just a couple of changes before merging:

href= refreshes the whole page including the navbar. You should change all instances of <Nav.Link href="/yourpath"> to <Nav.Link as={Link} to="/yourpath">.
This will make only the page content refresh rather than the whole page.
Could you run git rebase main and fix the conflict in Home.js?

Thank you so much Alessandro I made the changes that you point.


English missing from list of Languages

Rancher Version: master 10/10
Steps to Reproduce:

Click on the Language drop-down

Results: English is missing.  If I switch out of English, I can't go back.
Expected:  Should be able to switch to English.

Version - master 10/10
Verified fixed


Changed Required jQuery Version

The bower.json file should require a >=1.9.0 instead of being locked
into 1.9.x.  This was changed back to the way it was in 3.3.3.

Sorry, I did not see that there was already a pull request for this.  This needs to be changed because it is inhibiting upgrading from 3.3.3 to 3.3.4 with other dependencies.

Duplicate of https://github.com/Bttstrp/bootstrap-switch/pull/644


BatterySystem already has a Description property inherited from System

BatterySystem inherits from System
      "BatterySystem": {
        "allOf": [
          {
            "$ref": "#/components/schemas/System"
          },
        ....
        ]
     }

And from it, the Description property
  "System": {
    "type": "object",
    "description": "Generic abstract base class for generation/storage systems.",
    "properties": {
      "Description": {
        "$ref": "#/components/schemas/Description"
      },
      ...
    }
  },

But BatterySystem defines again the same property
      "BatterySystem": {
        "allOf": [
          {
            "$ref": "#/components/schemas/System"
          },
          {
            "type": "object",
            "description": "A battery system",
            "properties": {
              ...
              "Description": {
                "$ref": "#/components/schemas/Description"
              },
              ...
            }
          }
        ]
      },

This doesn't raise any problems with the OB's editor or the OpenAPI standard, but it does seem to be unnecessarily redundant.

@naulacambra which release of the taxonomy are you looking at? The 2207 release added the inheritance and removed direct use of the inherited elements. In the latest release and the development file, I'm not seeing Description both inherited and defined directly.

I see your point now. In System there is description (line 10166) and Description (line 10168). I'll have to look into this with someone more expert in json than I am. There isn't a conflict when we make instance files, so perhaps description (at the same level as properties is a keyword for processing the json file?
The remark about the 2207 release is comparing back to the 2202 release. In 2202, BatterySystem didn't inherit System so had Description (and other objects) on it explicitly. In 2207 we created System as a base class to be inherited by BatterySystem, PVSystem and others.

I'm sorry, I probably explained myself the wrong way.
I'm only talking about the Description object inside the properties attribute, not the description attribute itself (note the capitalization).
In the PR #73 Creates Generic System Object, as you said, the System schema was created to group the common properties between PVSystem and BatterySystem.
In this PR is shown that BatterySystem and PVSystem had Description as a property before the changes
PVSystem.Description

BatterySystem.Description

And after the changes, System has a Description property
System.Description

BatterySystem still has a Description property
BatterySystem.Description

But PVSystem doesn't have a Description property anymore
PVSystem

With this change, it seems that PVSystem stops having a Description property because now it inherits it from System (which makes all the sense) but BatterySystem still have the Description property defined.
Proposed solution: Remove Description from BatterySystem

@naulacambra thanks for being patient here! I see the duplicate Description values now. I overlooked them when I looked at the raw json. And, this has uncovered issues with our editor (openobediter.sunspec.org), which only shows the inherited Description, and our CI checks, which should have picked up this duplication.

This needs a fix in the ob-editor


 This will help facilitate the usage of the back button and provide more variations of sharable links.

I believe I got everything in this here https://github.com/AnubisLMS/Anubis/commit/e7b1336ddfdfeb087b0d66284a0d0117b4cf34d5
Reopen this if there is anything that I missed


2.1.0 dist folder doesn't contain updated code

👋
Looks like the dist js files in 2.1.0 don't contain the latest changes

Odd, because the .d.ts does have correct types.

Side note: have you considered publishing via GitHub actions so the build+publish is handled consistently rather than manually publishing? I'm happy to offer a PR for that if you'd like.
Thanks

ugh, i don't know why this keeps happening. Something is weird. I have a github action for publishing but it's erroring. Ill take a look at this and re-release.

No worries, it happens!
here's a copy of an action I use to publish to NPM - it creates the publish every time you create a GitHub release with the format of *.*.* (e.g. 2.1.0 - note that it uses the version in the package json though)
name: Release

on:
  push:
    tags:
      - "*.*.*"

jobs:
  publish:
    name: Publish package to npm
    runs-on: ubuntu-latest
    steps:
      - name: Checkout repository
        uses: actions/checkout@v3

      - name: Setup Node
        uses: actions/setup-node@v3
        with:
          node-version: 18
          cache: "npm"

      - name: Install dependencies
        run: npm ci

      - name: Build package
        run: npm run build

      - name: Publish package
        uses: JS-DevTools/npm-publish@v1
        with:
          token: ${{ secrets.NPM_TOKEN }}

Thanks, I have something very similar that auto-emits when creating a github release. Just need to figure out what's wrong with it

Take a look at 2.1.1!

Awesome - dist is updated now, thank you! 🚀


mods to get the process right for the featurebase server service.

Updated the help for setting up a service
IMPORTANT! AWOOGAH!!!
Someone will have to test the instructions on a Linux/MacOS environment because I don't have that at this end right now.
I'm especially interested in the settings in the new file /com-config-service-fb.md because the originals depend on featurebase.conf being in the OS /etc directory NOT where it is currently in /featurebase/opt
Similarly, the user molecula could be incorrect given we're on Featurebase 3+

I've now tested this locally and it works.


Issue966 timezone doc

Closes #966 => Expands documentation for desiredtz and configtz in the GGIR function help file, and for timenum format in part 5 time series both in the GGIR function help file and the main GGIR vignette.

Checklist before merging:

[x] Existing tests still work (check by running the test suite, e.g. from RStudio).
[ ] Added tests (if you added functionality) or fixed existing test (if you fixed a bug).
[ ] Updated or expanded the documentation.
[x] Updated release notes in inst/NEWS.Rd with a user-readable summary. Please, include references to relevant issues or PR discussions.
[ ] Added your name to the contributors lists in the DESCRIPTION file, if you think you made a significant contribution.

Codecov Report

Merging #969 (b405e29) into master (f256fb5) will decrease coverage by 0.09%.
The diff coverage is n/a.

:exclamation: Current head b405e29 differs from pull request most recent head c86f659. Consider uploading reports for the commit c86f659 to get more accurate results

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
@@            Coverage Diff             @@
##           master     #969      +/-   ##
==========================================
- Coverage   78.44%   78.35%   -0.09%
==========================================
  Files          99       99
  Lines       13350    13350
==========================================
- Hits        10473    10461      -12
- Misses       2877     2889      +12

see 4 files with indirect coverage changes


[Feature][Config] Rename config key

Search before asking

[X] I had searched in the feature and found no similar feature requirement.

Description
I think result_table_name is ambiguous with source_table_name and needs to be redefined to a suitable name. Do you have any suggestions?
Usage Scenario
No response
Related issues
No response
Are you willing to submit a PR?

[X] Yes I am willing to submit a PR!

Code of Conduct

[X] I agree to follow this project's Code of Conduct

source_table_name -> plugin_input_identifier
result_table_name -> plugin_output_identifier


transpile web modules

web modules are not transpiled. i know RN does not require it but some web packages have been doing this for web to make framework support easier

Hey @alidcast, Can you try the latest version of @dank-style/react?


Add docs for WATCH_NAMESPACE

Provide a description of what has been changed
Checklist

[x] Commits are signed with Developer Certificate of Origin (DCO)

Fixes https://github.com/kedacore/keda/issues/5670

/run-e2e internal


Successfully submit consent in ManageConsents journey

This completes the end to end ManageConsent journey.
Gillick competence is next to do in a follow-up, and then we can flip the feature flag and run the tests against the new journey to see what else needs aligning.

:shipit: I consent!


Make tests concurrent

Now we can have a quick overview for all the tests in all the instances.

thanks! We should just make sure the tests will run at a time that it won't be a problem that it will take more resources for the service (I am not sure what's the current timing, 4 in the morning?)

Yes, now they run at 4 in the morning (a part when someone, like me, retriggers them during the day). And they last till the morning. I hope, in this way, they should during the night.


Add datatype information to PycroData output

Easy way to access the dtype info especially field names for compound types.

Done


Parser fails on types like [A | B]

This task is automatically imported from the old Task Issue Board and it was originally created by Radosław Waśko.
Original issue is here.

Steps to reproduce
Create and run the following script:
from Standard.Base import all

foo : [Integer | Text] -> (Integer | Text)
foo v = v.at 0

main =
    IO.println <| here.foo [1,2,3]

Expected
1

Actual
Exception in thread "main" org.graalvm.polyglot.PolyglotException: java.lang.Error: Internal error: not all template segments were fully matched
	at org.enso.syntax.text.ast.meta.Builder.build(Builder.scala:89)
	at org.enso.syntax.text.prec.Macro$Transformer.$anonfun$run$2(Macro.scala:72)
	at org.enso.syntax.text.prec.Macro$Transformer.$anonfun$run$2$adapted(Macro.scala:68)
	at scala.collection.immutable.List.foreach(List.scala:333)
	at org.enso.syntax.text.prec.Macro$Transformer.run(Macro.scala:68)
(...) (truncated the rest)

Comments:
For the new parser, I added a test case for this in https://github.com/enso-org/enso/pull/3802; it parses correctly, but TreeToIr's handling of the case can't be tested yet, since the TreeToIr tests use the old parser as an oracle, and the old parser crashes in this case. (Keziah Wesley - Oct 17, 2022)

We shouldn't write in this form - should be Vector (Text | Number)


Adds example of session personas to UserInfo view in BrandGame

This PR also fixes a bug when removing metadata in the MetadataHandler.
The lifespanId was not being passed in correctly so only permanent records could be removed.

Seems that lifespanContext is just the id.
Left some suggestions so we just use lifespanId so it's less confusing.


Create lint rule to prevent passing only onClick to ActionList.Item

I would like to suggest that we create a lint rule that prevents using only onClick with ActionList.Item and that would auto-fix to use onSelect instead.
Context
I've recently fixed a number of accessibility bugs where behavior was only accessible with the mouse because we passed onClick to an action list item, instead of using the onSelect attribute.
Examples:

https://github.com/github/github/pull/333584/files#diff-e126663a316e199308eb3b48cc3837f0bad69f108e36538afc629f92051ac85bR210
https://github.com/github/github/pull/333599/files#diff-17bdd00340af50ddd6302d900b33310f25328339c12cfd22b9a68792e50fa76dR198
https://github.com/github/github/pull/333618/files#diff-75f8395823a4cf1bc91eee589beaa11d22c36c9ebf90ff5a352102f0a348b71eR105

While onClick is not completely invalid to put on ActionList.Item, if it is the only means by which an item can be activated, then it creates an inaccessible experience.
Using code search, I can also see some other code examples which might be problematic:

https://github.com/search?q=repo%3Agithub%2Fgithub+%2FActionList.Item\n.*onClick%2F&type=code
https://github.com/search?q=repo%3Agithub%2Fgithub+%2FActionList.Item.*onClick%3D%2F&type=code
https://github.com/search?q=repo%3Agithub%2Fgithub+%2FActionList.Item\n.*\n.*onClick%3D%2F&type=code
https://github.com/search?q=repo%3Agithub%2Fgithub+%2FActionList.Item\n.*\n.*\n.*onClick%3D%2F&type=code

Examples
Code that should be considered invalid by this rule:
<ActionList.Item onClick={...}>

Code that should be considered valid by this rule:
<ActionList.Item onSelect={...}>
<ActionList.Item onSelect={...} onClick={...}>

https://github.com/github/github/pull/336982 is merged in gh/gh
Next steps: upstream after iterating on feedback?

👋 @camchenry thanks for proposing this! Are you interested in upstreaming it to Primer?

@lesliecdubs Sure! If I have some time this week, I can work on upstreaming this rule.


search: 409 conflict when faceting a timestamp or timestamptz column

ERROR: 409
I got the following error message when I tried to facet based on a timestamp or timestamptz column. Please fix.
ResponseText: 409 Conflict
The request conflicts with the state of the server. invalid input syntax for type timestamp: "Invalid Date"
LINE 8: WHERE ((t0."Creation_Timestamp" >= 'Invalid Date'::timestamp...
^

TextStatus: error
ErrorThrown: Conflict
URL: https://dev.gpcrconsortium.org/ermrest/catalog/1/aggregate/A:=experiments:Experiment/Creation_Timestamp::geq::Invalid%20Date/$A/cnt:=cnt_d(Name)

@hongsudt In order to debug, please provide details where does it happen and the scenario used.

This is on dev@gpcr. Experiment table has a creation_timestamp which is defined as a timestamptz. I hid it from faceting but you could undo this to debug.

@hongsudt I have unhidden the column. The error as described above can not be recreated.
Looking at the table, it has 4 rows. There is a row having the NULL value for the Creation_Timestamp column; all the others have the same value 2016-08-22 21:25:38.479892-07.
I have detected based on this example, that for timestamps, as they are cast to dates, the maximum value needs to be incremented with 1 day.
Otherwise, the search results will not contain the records having the maximum value of the date.
I have fixed it through the commit 1ca259b9955d4a2d2913877ed9d8fe32e335ee6a. Anyway, that's something independent of the above mentioned bug.
Looks to me that somehow (?), the date widget had an error, and as a result, it was generating the "Invalid Date" value.

@akumar03 can you please review this fix.

Still see the error on dev:
https://dev.gpcrconsortium.org/chaise/search/#1/experiments:SOP?facets=(SOP:Last_Modified::geq::Invalid Date/SOP:Last_Modified::leq::Invalid Date)&layout=table&page=1
ResponseText: 409 Conflict
The request conflicts with the state of the server. invalid input syntax for type timestamp with time zone: "Invalid Date"
LINE 8: WHERE ((t0."Last_Modified" >= 'Invalid Date'::timestamptz))
^
TextStatus: error
ErrorThrown: Conflict
URL: https://dev.gpcrconsortium.org/ermrest/catalog/1/aggregate/A:=experiments:SOP/Last_Modified::geq::Invalid Date/$A/cnt_Title:=cnt(Title),cnt_Procedure:=cnt(Procedure)

@svoinea I am moving this to this week.

This is still a problem.. None of the timestamp seems to work. Please fix this asap.
https://demo2.gpcrconsortium.org/chaise/search/#1/experiments:experiment?facets=(experiment:last_modified_timestamp::geq::Invalid Date/experiment:last_modified_timestamp::leq::Invalid Date)&layout=table&page=1

Should be fixed now through the commit ee92efe07c744d0aab507a37db3f9959673986e6 in the chaise repository.
Currently testing can be performed using the https://dev.gpcrconsortium.org/~serban/chaise/search/#1/experiments:experiment link.

When the issue is ready to be reviewed, please move it to the review column and assign the reviewer.

It is not working on gpcr-dev. It no longer throws an error, but all the dates are greyed out except Nov 3 and 4. And when I selected a date, no filtering was applied.

@svoinea Your commit doesn't fix the problem.. See my previous comment.

I don't understand which are the expectations.
All the entries have a single date: November 3.
What filter do you want to apply? If in the From: widgetyou select 'November 4, you get zero results (because none of the timestamps is equal toNovember 4). Same thing happens when you select in the To:widgetNovember 3`.

I should be able to select an arbitrary begin date and end date and apply
that as a filter..
For example, I should be able to ask for everything from 10/15 to 11/31 and
that shouldn't fail.
On Mon, Nov 7, 2016 at 10:11 AM, svoinea<EMAIL_ADDRESS>wrote:

I don't understand which are the expectations.
All the entries have a single date: November 3.
What filter do you want to apply? If in the From: widgetyou select
'November 4, you get zero results (because none of the timestamps is equal
toNovember 4).
Same thing happens when you select in theTo:widgetNovember 3`.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/informatics-isi-edu/chaise/issues/545#issuecomment-258915132,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAZUDEpjO0TBfM_rfMps82b69mQ--5JCks5q72nugaJpZM4JqinR
.[image: Web Bug from
https://github.com/notifications/beacon/AAZUDD5VF0ViQO7trKnmOIt6F8fWWS1Xks5q72nugaJpZM4JqinR.gif]
{"api_version":"1.0","publisher":{"api_key":"
05dde50f1d1a384dd78767c55493e4bb","name":"GitHub"},"entity":
{"external_key":"github/informatics-isi-edu/chaise","
title":"informatics-isi-edu/chaise","subtitle":"GitHub
repository","main_image_url":"https://cloud.githubusercontent.com/assets/
143418/17495839/a5054eac-5d88-11e6-95fc-7290892c7bb5.png","
avatar_image_url":"https://cloud.githubusercontent.com/
assets/143418/15842166/7c72db34-2c0b-11e6-9aed-b52498112777.png
","action":{"name":"Open in GitHub","url":"https://github.
com/informatics-isi-edu/chaise"}},"updates":{"snippets":[{"
icon":"PERSON","message":"@svoinea in #545: I don't understand which are
the expectations.\r\nAll the entries have a single date: `November
3`.\r\nWhat filter do you want to apply? If in the `From:` widget` you
select 'November 4, you get zero results (because none of the timestamps is
equal to `November 4`).\r\nSame thing happens when you select in the `To:`
widget `November 3`.\r\n"}],"action":{"name":"View Issue","url":"
https://github.com/informatics-isi-edu/chaise/issues/545#
issuecomment-258915132"}}}


Merge Branch AboutUs

Added an arbitrary role to test if my email settings were updated correctly.

Could you change the last few commits to author under this main account? You might want to either undo the commits, or more easily: squash them together and re-author under this new email https://stackoverflow.com/questions/5189560/how-do-i-squash-my-last-n-commits-together


Sendtarget causes both the current character and the intended character to do the command

On Asen (72 PLD) with Chaoticunreal (75 rdm) with the alias ch sta Chaoticunreal defined.
doing //ch cure4 me (or //ch //cure4 me) causes both characters to cast cure4 on Asen.
Also happened with curingwaltz so I don't think the command effects it

I was finally seeing this effect myself today. Haven't had a chance to investigate it yet.

Any news on this?

Unfortunately, that "today" in my last message was the only time I experienced it. I wasn't able to reproduce it when I went to test it, and haven't noticed it again since...
I have noticed a similar bug though when using other addons that inject comments, such as AutoRA. If it triggers while doing a <stpc> or other subtargeting command, it disrupts the state and prevents the command from being processed by the addon.
Only thing I can ask is about other addons you might be using when encountering the bug.

Looks like it only happens with gearswap so I'll go through that and see if I can figure out wth is going on.

Closing the loop on this here looks like plugin_manager just loads the plugin/addons in alphabetical order so even having gearswap at the bottom of the list it was still being loaded before sendtarget.  Reloading gearswap fixes the problem.


Unpin conda for benchmarks when asv gets fixed

📰 Custom Issue

Currently, conda needs to be pinned to get asv to work. This happens here: https://github.com/SciTools-incubator/iris-esmf-regrid/blob/7696d493215db7bdbc061c1be3682c505d9b4a03/noxfile.py#L373
When this asv issue gets fixed, this line should be removed: https://github.com/airspeed-velocity/asv/issues/1396

closed by #361


Support populateCache as a function

We added the populateCache option (as well as other options for optimistic UI) recently, a normal use case looks like the following:
mutate(addTodo(todo), {
  optimisticData: [...todos, todo],
  populateCache: true,
  rollbackOnError: true,
  revalidate: false,
})

Say that the addTodo calls the POST /api/todos endpoint, which returns the full, updated list.
Transform Before Populating the Cache
However, it's also common that the POST /api/todos endpoint only returns the new added item, not the full list. So naturally the next step is to support populateCache as a synchronous function that transforms the returned data before writing back to the cache:
mutate(addTodo(todo), {
 optimisticData: [...todos, todo],
 populateCache: addedTodo => [...todos, addedTodo],
 rollbackOnError: true,
 revalidate: true,
})

Note that revalidate: true is needed now.
Also populateCache is synchronous only to avoid possible race conditions.
Opt-out
With this design, if an error is throw when calling populateCache, we can skip populating the cache for this specific mutation.
This enables a way to conditionally populating the cache.

Hi @shuding, thank you for this improvement!
I have a similar need with the library. I would like to use optimisticData as a function, that takes as argument the current store value. It would allow me to extract the mutation functions more easily out of the component that uses the store.
Could you do that ?
Thanks!

that takes as argument the current store value

@ludovic-montel For this do you mean the current returned data property from swr hook? Guess it would also be possible to evaluate outside of mutate function, since swr has returned mutate and data at the same time.
Or are you using the global mutate building the function outside of components and store?

I think then the populateCache option needs to accept the current data as well, something like:
mutate(addTodo(todo), {
  optimisticData: todos => [...todos, todo],
  populateCache: (addedTodo, todos) => [...todos, addedTodo],
  rollbackOnError: true,
  revalidate: true,
})

So this can be wrapped in an event callback which is totally pure and doesn't depend on the hook state.

Yeah building up hook with useSWRConfig().mutate could be generic and independent from the specific hook key/data

Or are you using the global mutate building the function outside of components and store? I think this case sounds fair

@huozhi yes, exactly, it would allow to use the global mutate function with the optimisticData, it would be great!

This can be a great separate feature request 👍

Thanks for your answers, I just added a feature request in the Discussion of the repository (https://github.com/vercel/swr/discussions/1850)