Webhook Sentry might be deleting request body somehow?

This one is a long shot, but we're pretty baffled about it, so I'm writing it down here in case it rings a bell with you, @juggernaut, or perhaps somebody else.
We're using Webhook Sentry in a k8s environment. Every so often, a request goes into Webhook Sentry with a request body, but arrives at our webhook endpoint without it. Somewhere along the line the request body is being stripped.
The full process looks like this:

Our application layer creates a webhook POST with a body.
An assertion is checked just before the next step, to ensure the request has a body.
The request is posted to Webhook Sentry, which proxies it to our webhook endpoint of a different service.
Our webhook endpoint checks request.json, but it's empty.
Our webhook endpoint therefore crashes and throws an HTTP error.
The application layer from step 1 catches the error and retries the webhook.
It works.

We aim to send a high volume of webhook events, so we want to track this down. After having this error happen a few times, we put an assertion at step 2 above to make sure that we weren't sending empty bodies. Today, we had the error happen again, meaning that the assertion passed, but somehow the webhook endpoint didn't get data on step 4.
I'm entirely open to the possibility that there's something in our network layer causing this, but:

It would be weird for an encrypted request to be modified in transit.
We don't have anything particularly weird in our network layer.

We're at a bit of a loss. Data is going into the network OK—it passes the assertion—but is somehow vanishing along the way. Does this make any sense to you or maybe ring any bells? Is it something that a bug in Webhook Sentry could cause?
If you have any ideas for how to look into this, we'd be happy to try to help debug/fix it. We're motivated on this one! Thank you!
We're tracking the issue here.

That seems strange @mlissner . I haven't encountered this issue so far. So that I can attempt to reproduce on my end, can you please provide the following info:

Headers and request params passed to webhook-sentry
Approximate size of request body
webhook-sentry config
Does it happen only under load, or even when there is zero to no load?

Thanks!

Thanks @juggernaut  here are the details about the issue:

1.- Headers and request params passed to webhook-sentry:

These are the headers we're sending to webhook proxy:
"Content-type": "application/json"
"Idempotency-Key": An uuid string used by the client to identify the uniqueness of the request.
"X-WhSentry-TLS": "true"

And the request is made with the following params:
We're using Python requests module:
proxy_server = {
        "http": "http://cl-webhook-sentry:9090",  #The proxy URL.
    }
requests.post(
            url,
            proxies=proxy_server,
            json=json_data,
            timeout=(3, 3),
            stream=True,
            headers=headers,
            allow_redirects=False,
        )

The url param here is changed in order to use the http protocol as described in documentation

Approximate size of request body

Here are some sizes of the request body from different events that failed:
8,551 bytes
90,293 bytes
18,314 bytes
10,252 bytes
12,278 bytes
11,841 bytes
In the end, events usually succeed after some retries.
I checked many events in which the delivery succeeded on the first attempt, and the average size is approximately 3,000 bytes. Interestingly, I've not encountered an event where this issue occurred with a size of less than 8,000 bytes. It is uncertain whether this is merely a coincidence.

webhook-sentry config

We're using the docker image juggernaut/webhook-sentry:v1.1.0 with the default config.go file.
In case is helpful this is the k8s config:
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: cl-webhook-sentry
  namespace: court-listener
  labels:
    service: cl-webhook-sentry
spec:
  replicas: 2
  selector:
    matchLabels:
      service: cl-webhook-sentry
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        service: cl-webhook-sentry
    spec:
      nodeSelector:
        eks.amazonaws.com/nodegroup: static-egress-node-group
      containers:
        - name: webhook-sentry
          image: juggernaut/webhook-sentry:v1.1.0
          imagePullPolicy: IfNotPresent
          ports:
           - containerPort: 9090
          livenessProbe:
            periodSeconds: 5
            httpGet:
              path: /metrics
              port: 2112

---
apiVersion: v1
kind: Service
metadata:
  name: cl-webhook-sentry
  namespace: court-listener
  labels:
    service: cl-webhook-sentry
spec:
  selector:
    service: cl-webhook-sentry
  ports:
    - protocol: TCP
      port: 9090
      targetPort: 9090

Does it happen only under load, or even when there is zero to no load?

At the moment, we are not experiencing a high load, as we are sending no more than 15 events per hour. Today's error occurred during a period of low load; it was the only event sent at that time. There was a successful event two minutes prior and another one ten minutes later.
Finally, I'm not sure if this might be related but I noticed a similar issue sending events to an HTTPS ngrok endpoint using webhook-sentry with X-WhSentry-TLS header set to True, the headers are properly received but the data is empty.
Thank you!

Thanks for the details @albertisfu ! I'll take a look tomorrow and get back to you. Also, just to confirm, you're not using ngrok in your production setup right? The issue with ngrok is when you were doing dev stuff locally I presume?

Thank you! yes, ngrok is not used in production, I noticed the issue doing some dev testing.

@albertisfu @mlissner I tried reproducing with different request body sizes (and with ngrok in the request path), but could not reproduce it. I was using a flask app as the target server. However, I did notice that webhook-sentry is always sending a Transfer-Encoding: chunked instead of the Content-Length header even when its available on the input request.
Going through your linked ticket, it appears you are using Django which according to cursory googling may have issues dealing with chunked encoding (https://stackoverflow.com/questions/12091067/handling-http-chunked-encoding-with-django). Can you share a little bit more about your setup? How does the request path look like, right from the origin to your target webhook receiver? I'm wondering if something along the path "fixes up" the chunked encoding header, but not in all cases, which may explain the intermittent behavior you are seeing. If you an provide a condensed version of your django receiver endpoint that would be a great help as well.
I have a fix for copying over the Content-Length header, but I would rather reproduce this issue first to understand the root cause, than just apply a fix without confirming it actually fixes the issue.

Thanks so much for supporting your work like this, @juggernaut.
While Alberto is working on reproducing this, I did a bunch more googling. Almost everything I could was, really old, but I did find a post from earlier this year in the gunicorn issue tracker. It says:

With gunicorn 20.1.0 - when it receives a chunked request - the body is available to the application from wsgi.input but the headers are untouched: the application sees no CONTENT_LENGTH header.
Is this the intended behaviour? Django for example will return an empty body in this case.

To which a "contributor" to gunicorn replies:

Yes, this is intentional. Transfer-Encoding: chunked must not set a content length. This is what the wsgi.input_terminated extension to the spec is for, Gunicorn and other WSGI servers set it to indicate that they handle terminating the stream and that it's safe to read. Django does not use this, it does not support streaming request bodies.

So that feels like a smoking gun, but it's unclear why it only fails sometimes.

Our full network architecture is:

Our main server uses Python requests to send a POST request (webhook event) to Webhook Sentry (via the code Alberto posted above).
Webhook Sentry forwards the request to our other server over the Internet.
The other server is behind an AWS Application Load Balancer (AWS ALB).
The ALB forwards the request to a kubernetes pod running Gunicorn.
Gunicorn, using the sync workers, sends the request to Django.
Django processes the request, and returns an error.

I don't think it matters, but the main server, Webhook Sentry, and the recipient server are all in the same k8s cluster (though with different namespaces).

As for the steps to reproduce that Alberto's working on, it's tough because he needs to get Gunicorn set up with a TLS certificate to create a minimal example, but he's working on it.
Thank you again. Feels like you've got a pretty good theory here. Hopefully Alberto can prove it.

@juggernaut, we have created a repository where we've reproduced the error within a Docker environment, making it easy to test and evaluate:
https://github.com/albertisfu/test-wh-sentry
The webhook client uses Django and Gunicorn, while send-request.py employs Webhook Sentry as a proxy.
There are some instructions in the repo and you'll be able to view logs similar to these:

Initially, I believed the issue occurred exclusively when using X-WhSentry-TLS, but it also happens for http targets, as in this example.
Please let us know if you require any further information or how can we help.
Thanks.

Alberto and I talked about this a bit today and the one remaining piece of the puzzle we don't understand is why it only happens some of the time. Our best theory is that:

Webhook Sentry provides the headers you mentioned, @juggernaut.
Sometimes routers along the way tweak the headers to simplify the request, perhaps particularly on small requests.
Those tweaked requests fail.
We do automatic retries.
Different routers are used to forward the packets.
Those routers don't tweak the message.
It works.

Thanks @albertisfu and @mlissner for creating the docker environment! I was able to reproduce the issue and test the fix as well. It was as expected -- the Content-Length header was not being passed along by the proxy. This will be addressed by https://github.com/juggernaut/webhook-sentry/pull/16 . Once it's merged I'll cut a new release.
@mlissner I would agree with your assessment of why it happens only some times. My money is on some instances of the ALB "fixing-up" the content length and some not. We'd need a packet capture on either side of the ALB to figure this out, but probably not worth the trouble since we have the fix now.
Another thing to mention -- you'll want to remove stream=True from your requests.post(..) call on the client. From an SO answer, stream=True sends Transfer-Encoding: chunked (but in my testing, I only saw Content-Length even when stream=True), so you'd want to disable that out of caution since your server can't handle it.

Release v1.1.1 is out with the fix, please verify that it fixes the issue!

Lovely. It's deployed. We won't know for some time if this worked, but we'll keep the issue open on our end and will report results in a few weeks.
Thank you again for your work supporting this awesome tool. It's really nice to benefit from the security it provides and there's nothing else like it that I've found.

Just to close this loop, it's been about a month since we deployed the updated webhook-sentry pod and we haven't had this issue since. Yay!

Great news @mlissner !


Fix: apply style utility for unstable anchorPosition

Details

This PR fixes the issue of floating dropdown menu position, of the group avatar, when resizing the browser.
Fixed Issues

$ https://github.com/Expensify/App/issues/40944
PROPOSAL: https://github.com/Expensify/App/issues/40944#issuecomment-2077168008
Tests

Click on FAB > Start chat > Add email
Click on the header > Upload photo
Minimize the webpage > Click on the avatar to display the dropdown
Maximize the webpage.
Verify that the dropdown align with the edit photo button and doesn't float in the composer

[x] Verify that no errors appear in the JS console

Offline tests

QA Steps

Click on FAB > Start chat > Add email
Click on the header > Upload photo
Minimize the webpage > Click on the avatar to display the dropdown
Maximize the webpage.
Verify that the dropdown align with the edit photo button and doesn't float in the composer

[x] Verify that no errors appear in the JS console

PR Author Checklist

[x] I linked the correct issue in the ### Fixed Issues section above
[x] I wrote clear testing steps that cover the changes made in this PR

[x] I added steps for local testing in the Tests section
[x] I added steps for the expected offline behavior in the Offline steps section
[x] I added steps for Staging and/or Production testing in the QA steps section
[x] I added steps to cover failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
[x] I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)
[x] I tested this PR with a High Traffic account against the staging or production API to ensure there are no regressions (e.g. long loading states that impact usability).

[x] I included screenshots or videos for tests on all platforms
[x] I ran the tests on all platforms & verified they passed on:

[x] Android: Native
[x] Android: mWeb Chrome
[x] iOS: Native
[x] iOS: mWeb Safari
[x] MacOS: Chrome / Safari
[x] MacOS: Desktop

[x] I verified there are no console errors (if there's a console error not related to the PR, report it or open an issue for it to be fixed)
[x] I followed proper code patterns (see Reviewing the code)

[x] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick)
[x] I verified that the left part of a conditional rendering a React component is a boolean and NOT a string, e.g. myBool && <MyComponent />.
[x] I verified that comments were added to code that is not self explanatory
[x] I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
[x] I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method

[x] If any non-english text was added/modified, I verified the translation was requested/reviewed in #expensify-open-source and it was approved by an internal Expensify engineer. Link to Slack message:

[x] I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
[x] I verified any copy / text that was added to the app is grammatically correct in English. It adheres to proper capitalization guidelines (note: only the first word of header/labels should be capitalized), and is either coming verbatim from figma or has been approved by marketing (in order to get marketing approval, ask the Bug Zero team member to add the Waiting for copy label to the issue)
[x] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
[x] I verified the JSDocs style guidelines (in STYLE.md) were followed

[x] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
[x] I followed the guidelines as stated in the Review Guidelines
[x] I tested other components that can be impacted by my changes (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar are working as expected)
[x] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
[x] I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
[x] I verified that if a function's arguments changed that all usages have also been updated correctly
[x] If any new file was added I verified that:

[x] The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory

[x] If a new CSS style is added I verified that:

[x] A similar style doesn't already exist
[x] The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(theme.componentBG))

[x] If the PR modifies code that runs when editing or sending messages, I tested and verified there is no unexpected behavior for all supported markdown - URLs, single line code, code blocks, quotes, headings, bold, strikethrough, and italic.
[x] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
[x] If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
[x] If the PR modifies a component or page that can be accessed by a direct deeplink, I verified that the code functions as expected when the deeplink is used - from a logged in and logged out account.
[x] If the PR modifies the UI (e.g. new buttons, new UI components, changing the padding/spacing/sizing, moving components, etc) or modifies the form input styles:

[x] I verified that all the inputs inside a form are aligned with each other.
[x] I added Design label and/or tagged @Expensify/design so the design team can review the changes.

[x] If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
[x] If the main branch was merged into this PR after a review, I tested again and verified the outcome was still expected according to the Test steps.

Screenshots/Videos

Android: Native

NA

Android: mWeb Chrome

NA

iOS: Native

NA

iOS: mWeb Safari

NA

MacOS: Chrome / Safari

https://github.com/Expensify/App/assets/12425932/270a4438-68fc-4157-b11f-02e1a324fec2

MacOS: Desktop

NA

What are we thinking is the ideal location for these popovers though? Is it perfectly centered under the avatar? I ask because it does not look perfectly centered here:

But it does look centered here:

@shawnborton I think perfectly centered makes sense. I did some measuring, and it looks like they're maybe off by 1px?
I'm wondering if the scrollbar in your first screenshot is throwing off the visual alignment.

(Side note: looks like these menus are using an incorrect border radius 😑)

If they are off by just 1px, than I bet the border width is is what's throwing things off?
Good catch on the border radius too, maybe we can create a follow up issue to fix that?

I also think we should do centered under the avatar as long as it doesn't go less than 20px from the edge

Reviewer Checklist

[ ] I have verified the author checklist is complete (all boxes are checked off).
[ ] I verified the correct issue is linked in the ### Fixed Issues section above
[ ] I verified testing steps are clear and they cover the changes made in this PR

[ ] I verified the steps for local testing are in the Tests section
[ ] I verified the steps for Staging and/or Production testing are in the QA steps section
[ ] I verified the steps cover any possible failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
[ ] I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)

[ ] I checked that screenshots or videos are included for tests on all platforms
[ ] I included screenshots or videos for tests on all platforms
[ ] I verified tests pass on all platforms & I tested again on:

[ ] Android: Native
[ ] Android: mWeb Chrome
[ ] iOS: Native
[ ] iOS: mWeb Safari
[ ] MacOS: Chrome / Safari
[ ] MacOS: Desktop

[ ] If there are any errors in the console that are unrelated to this PR, I either fixed them (preferred) or linked to where I reported them in Slack
[ ] I verified proper code patterns were followed (see Reviewing the code)

[ ] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick).
[ ] I verified that the left part of a conditional rendering a React component is a boolean and NOT a string, e.g. myBool && <MyComponent />.
[ ] I verified that comments were added to code that is not self explanatory
[ ] I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
[ ] I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method
[ ] I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
[ ] I verified any copy / text that was added to the app is grammatically correct in English. It adheres to proper capitalization guidelines (note: only the first word of header/labels should be capitalized), and is either coming verbatim from figma or has been approved by marketing (in order to get marketing approval, ask the Bug Zero team member to add the Waiting for copy label to the issue)
[ ] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
[ ] I verified the JSDocs style guidelines (in STYLE.md) were followed

[ ] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
[ ] I verified that this PR follows the guidelines as stated in the Review Guidelines
[ ] I verified other components that can be impacted by these changes have been tested, and I retested again (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar have been tested & I retested again)
[ ] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
[ ] I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
[ ] If a new component is created I verified that:

[ ] A similar component doesn't exist in the codebase
[ ] All props are defined accurately and each prop has a /** comment above it */
[ ] The file is named correctly
[ ] The component has a clear name that is non-ambiguous and the purpose of the component can be inferred from the name alone
[ ] The only data being stored in the state is data necessary for rendering and nothing else
[ ] For Class Components, any internal methods passed to components event handlers are bound to this properly so there are no scoping issues (i.e. for onClick={this.submit} the method this.submit should be bound to this in the constructor)
[ ] Any internal methods bound to this are necessary to be bound (i.e. avoid this.submit = this.submit.bind(this); if this.submit is never passed to a component event handler like onClick)
[ ] All JSX used for rendering exists in the render method
[ ] The component has the minimum amount of code necessary for its purpose, and it is broken down into smaller components in order to separate concerns and functions

[ ] If any new file was added I verified that:

[ ] The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory

[ ] If a new CSS style is added I verified that:

[ ] A similar style doesn't already exist
[ ] The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(theme.componentBG)

[ ] If the PR modifies code that runs when editing or sending messages, I tested and verified there is no unexpected behavior for all supported markdown - URLs, single line code, code blocks, quotes, headings, bold, strikethrough, and italic.
[ ] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
[ ] If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
[ ] If the PR modifies a component or page that can be accessed by a direct deeplink, I verified that the code functions as expected when the deeplink is used - from a logged in and logged out account.
[ ] If the PR modifies the UI (e.g. new buttons, new UI components, changing the padding/spacing/sizing, moving components, etc) or modifies the form input styles:

[ ] I verified that all the inputs inside a form are aligned with each other.
[ ] I added Design label and/or tagged @Expensify/design so the design team can review the changes.

[ ] If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
[ ] If the main branch was merged into this PR after a review, I tested again and verified the outcome was still expected according to the Test steps.
[ ] I have checked off every checkbox in the PR reviewer checklist, including those that don't apply to this PR.

Screenshots/Videos

Android: Native

Android: mWeb Chrome

iOS: Native

iOS: mWeb Safari

MacOS: Chrome / Safari

MacOS: Desktop

If they are off by just 1px, than I bet the border width is is what's throwing things off?
Good catch on the border radius too, maybe we can create a follow up issue to fix that?

Yeah I think we can create a follow up issue for that. Small css tweak so should probably be priced lower than a standard issue I think.

@dragnoir Please address above comment

@dannymcclain would you mind making a follow up issue for the centering bug?

:hand: This PR was not deployed to staging yet because QA is ongoing. It will be automatically deployed to staging after the next production release.

🚀 Deployed to staging by https://github.com/puneetlath in version: 1.4.77-0 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅

🚀 Deployed to production by https://github.com/puneetlath in version: 1.4.77-11 🚀

platform
result

🤖 android 🤖
failure ❌

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅

🚀 Deployed to production by https://github.com/puneetlath in version: 1.4.77-11 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅


ERROR::FREETYPE: Failed to load font

Hi, I have installed freetype i'm still getting this error.
./sketch rendering black screen
I'm using Ubuntu 20.04

I found the cause of this -- under processing.h line 1183 there is a hardcoded path. I changed it on my machine. The default font is also initialized here.

Hello, I'm not have enougth time to develop this library. And yes font path is hardcoced. Need to specify default font depends on current system. Sorry for long answer...

Fixed in ebfdcf8


Container image is missing flock util

The Dockerfile is based on alpine which contains a very old flock version (1.25).
We need > 2.25 flock to match the hyperkube image.

Figured out this is not needed.


Unable to resolve some modules: spawn-sync

I'm running Meteor <IP_ADDRESS> with fourseven:scss v3.10.1 and everytime my application runs I get the following error:
Unable to resolve some modules:

  "spawn-sync" in /home/robin/moonkin/bazaar/orbiter/node_modules/cross-spawn/index.js (web.browser)

If you notice problems related to these missing modules, consider running:

  meteor npm install --save spawn-sync

Running the advised command doesn't change anything. As far as I know it's node-scss that depends on this package (or whatever the package fourseven:scss depends on is called exactly).
Any ideas on how I could fix this?

i got same issue,
but I didn't follow up that advise cause it's not clear.
Any idea?

Can you retry with Meteor <IP_ADDRESS>+ and latest fourseven:scss

Still getting the same issue. Had ignored it for long, until today, and saw this issue open.
Meteor 1.7, fourseven:scss updated.


Create farmint-start-up-cost-estimation calculator module

This issue isn't actually important for a long time. Closing for now.


fix: bad conditional

What this PR does / why we need it:

JIRA ID: XX-XX

Notes for your reviewer:

:tada: This PR is included in version 1.23.1 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Should we setup Docker?

@is-it-ayush @DaatUserName @MultiEmail/backend-developers  Should we setup Docker for the backend?

Yes, it'd be good to use Docker. +1

Yes

Yes, It's nice to add docker in our project.

sure

Yes


🚀 Feature: Add Discord message sender with webhook - Node.js

🔖 Feature description
Task is to implement the Discord message sender with webhook function in Node.js.
Tasks summary:
Fork & clone Appwrite's templates repository
Create a branch feat-implement-discord-message-sender-webhook-with-nodejs
Write code for the function in folder node/discord-message-sender/.
Test the function using Appwrite, provide a video.
Write a README.md explaining what the function does, how to use it, and customize it.
Submit a pull request to our templates repository
🎤 Pitch
A function that would send a message to a specific channel in a specific discord server via webhook.
Some use cases:

Manual communication via function with the discord server (Custom Announcements, for example)
Sending appwrite project stats either manually or via cron task
Sending information from 3rd party APIs

👀 Have you spent some time to check if this issue has been raised before?

[X] I checked and didn't find similar issue

🏢 Have you read the Code of Conduct?

[X] I have read the Code of Conduct

@loks0n @Meldiron
If this is something you would like to add to the templates, I'd love to work on it.
Possibly during hacktoberfest as well :)
Would love to see what you think on this one too, along with the Slack one

hey @Meldiron can i work on this?

@loks0n @Meldiron
I would like to work on this. Can you assign it to me?

Hi @mbos2 we already have the Discord bot template. It is similar to your suggestion, let me know if that works for you. :)


Provide some tidy tools functionality

From one review:

Notes on the implications of focusing on "tidy":
When I saw the name of the package, I thought that there was going to be something like:
find_stations(bbox=c(w,x,y,z)) %>%
  get_data(parameter="flow") %>%
  calc_stats("annual")

I see what was meant is that it converts the data from messy to tidy....but just a side not that it would be cool get some hydro tidy tools too....maybe another day!
https://bookdown.org/rdpeng/RProgDA/gaining-your-tidyverse-citizenship.html

This could be done is a few ways. Possibly something like this:
CODE TO GET A VECTOR OF STATIONS %>%
map(DLY_FLOWS)

Or possible the dly_flows function could be modified such that it accepts a vector in a pipe for the STATION_NUMBER argument.

This has been provided using the pull() function as a link. For example this now works:
> search_stn_name("sam") %>%
   pull(STATION_NUMBER) %>%
   hy_daily_flows()
No start and end dates specified. All dates available will be returned.
All station successfully retrieved
# A tibble: 77,681 x 5
   STATION_NUMBER       Date Parameter Value Symbol
            <chr>     <date>     <chr> <dbl>  <chr>
 1        08HD004 1953-04-01      FLOW    NA   <NA>
 2        08HD004 1953-04-02      FLOW    NA   <NA>
 3        08HD004 1953-04-03      FLOW    NA   <NA>
 4        08HD004 1953-04-04      FLOW    NA   <NA>
 5        08HD004 1953-04-05      FLOW    NA   <NA>
 6        08HD004 1953-04-06      FLOW    NA   <NA>
 7        08HD004 1953-04-07      FLOW    NA   <NA>
 8        08HD004 1953-04-08      FLOW    NA   <NA>
 9        08HD004 1953-04-09      FLOW    NA   <NA>
10        08HD004 1953-04-10      FLOW    NA   <NA>
# ... with 77,671 more rows


Fixed bug where it was not possible to specify the connection to use

See issue #62
I added possibility to specify the connection (same as in core \Illuminate\Validation\Validator::validateUnique() method)

parseTable() is available from L5.1 and above. Fixed this.

@LittleBigDev I did a long overdue rewrite of the package, but I will look into rebasing this on master (and adding the new tests we need) in the next few days.

Oh ok then. I had no time for this anyway. Thx a lot.

Is this fixed?

Yes, here : https://github.com/jankivi/uniquewith-validator/commit/a12d4e15c071f043d57415143d599f0d159a69af

still relevant for me...
would be good to have this fix because I have same issue with different database connections

3 months passed, is there anything i could do to see this PR merged?

@ReDev1L you can also test this new PR I've created. The old PR was based on a version prior to complete rewrite, so I had to create a new one and never got around to it.


[$250] Task - Task details become blank after changing title or description several times

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

Version Number: 1.4.59-0
Reproducible in staging?: Y
Reproducible in production?: Y
If this was caught during regression testing, add the test name, ID and link from TestRail: N/A
Issue reported by: Applause - Internal Team
Action Performed:

Go to https://staging.new.expensify.com/
and log in
Open any chat and create a tasking,
Open the task details
Change the title or description several times

Expected Result:
The task details with the updated title or description are displayed every time the changes are made
Actual Result:
After updating the task title or description the task details become blank
Workaround:
Unknown
Platforms:

Which of our officially supported platforms is this issue occurring on?

[x] Android: Native
[ ] Android: mWeb Chrome
[ ] iOS: Native
[ ] iOS: mWeb Safari
[x] MacOS: Chrome / Safari
[ ] MacOS: Desktop

Screenshots/Videos
Add any screenshot/video evidence

https://github.com/Expensify/App/assets/78819774/1696fa18-75a4-4f46-b4c5-d2c9997e0511
View all open jobs on GitHub
Upwork Automation - Do Not Edit

Upwork Job URL: https://www.upwork.com/jobs/~0169f40ad10a6d41aa
Upwork Job ID:<PHONE_NUMBER>875103744
Last Price Increase: 2024-04-05

@mallenexpensify  FYI I haven't added the External label as I wasn't 100% sure about this issue. Please take a look and add the label if you agree it's a bug and can be handled by external contributors

Well... that's strange.  I was able to reproduce on staging.new.expensify.com in Chrome.  I was not able to reproduce on desktop. @dukenv0307 , please comment if you don't think this can be external.
https://github.com/Expensify/App/assets/22508304/09e66a12-8769-47b8-8921-5180d8b3092c

Thanks @dukenv0307 , Added retest weekly

Issue not reproducible during KI retests. (First week)

Gonna close as not reproducible. Comment/reopen if needed.


Specify robot configuration space and constrains for which the library works

Hi @singhbal-baljinder, if I am not wrong, the library works only for fixed-base, open-chain robots. If it is so, it would be nice to specify that in the repo README

Probably it is sufficient to add a few words in the initial sentence of the README, what do you think @singhbal-baljinder ?

@DanielePucci In #8  I have added some more info in the README, let me know if you think it is sufficient.

LGTM! @singhbal-baljinder

LGTM! @singhbal-baljinder

Perfect, then I'll close the issue.


v13.11.0: install script not working: missing binary distribution archives

I am reporting:

[x] a bug or unexpected behavior
[ ] general feedback
[ ] feature request
[ ] security issue

Bug Report
Please give us details about your installation to assist you. Run dat -v to see the version of Dat you are using.

Operating system: Linux
Node Version: 10.5.0
Dat Version: 13.11.0

Expected behavior
dat installed without using npm.
Actual behavior
curl returns a "404 Not Found" for the binary distribution archive.
macOS and Windows distribution archives missing too.
Debug Logs
$ wget -qO- https://raw.githubusercontent.com/datproject/dat/master/download.sh | bash
Fetching latest Dat release version from GitHub
Downloading https://github.com/datproject/dat/releases/download/v13.11.0/dat-13.11.0-linux-x64.zip

curl: (22) The requested URL returned error: 404 Not Found
============
Error: download failed

Ah, thanks!
Looks like the builds didn't get posted because tests are having issues right now. Will see if I can do it manually.

Thanks for reporting, should be fixed with the latest release: https://github.com/datproject/dat/releases/tag/v13.11.2.
Let us know if you have more issues installing.


detection of license for ua-parser-js is MIT instead of AGPL-3.0-or-later

When cloning https://github.com/faisalman/ua-parser-js/
git clone https://github.com/faisalman/ua-parser-js.git --depth=1

then
cd ./ua-parser-js

and running
npx @cyclonedx/cdxgen --type npm

I get a generated sbom that says the license is MIT but the license is AGPL-3.0-or-later

Interesting! It is taking the last data from npmjs instead of package.json. In enterprise environments, most package.json files might have Unlicense or some other unreliable id, so we generally do not rely on that file. Is this a matter of having a new release with the new license? Also, I am unsure if the existing published versions could be retroactively changed from MIT, but I am not a lawyer.

The existing published is indeed MIT (@prabhu thank you for spotting that). In this case we are more interesting in repository source of truth. For example assume the repo https://github.com/faisalman/ua-parser-js.git was a company/client repo and we are running cdxgen on it. Trying to generate the sbom from the repo source.

Looks like even with FETCH_LICENSE=0 it still returns MIT.

The repository is unreleased main branch mostly, while the license data in the sbom is specific to the released version. Have you tried scancode to see if it returns uptodate information?

I think the difference is app/project level vs. third party dependency. In the root of a repo the app/project doesn't need to use the registry to get info as it may not yet be published, where as it's third party dependencies and transitives are.

For app/project stuff I don't think the default should be to look to the registry and instead should look to its files available.


Deploying behind a reverse proxy

Hi Identity Server 4 team,
I'm currently deploying an Identity Server 4 installation. This is my first foray in to the brave new world of .NET Core, so I read and followed the advice of the .NET Core docs on publishing to a linux production environment which suggests deploying behind an nginx reverse proxy and using it for tasks like SSL termination, since Kestrel isn't built for that.
As a result, the outside world communicates with my installation through https://idsrv and in turn nginx reverse proxies that to http://localhost:5000.
Having looked through the source, unless I've missed something, it appears Identity Server 4 always uses the values in the current HttpContext to create URL's. Thus, although I'm visiting https://idsrv, the discovery document (for example) is sending back http://idsrv/connect/auth.
What would your advice be in this scenario? My solution so far has been to force HTTPS in my nginx config, and to change the scheme in the HttpContext at the start of a request (see below), but this feel really hacky to me.
// Startup.cs
if((new string[]{"production", "staging"}).Contains(env.EnvironmentName))
{
    app.Use(ForceSsl);
}

..

private static RequestDelegate ForceSsl(RequestDelegate next)
{
    return async ctx => {
        ctx.Request.Scheme = "https";
        await (next(ctx));
    };
}
``

The UseIISIntegration you typically see, like this:
var host = new WebHostBuilder()
                .UseKestrel()
                .UseUrls("http://localhost:5000")
                .UseContentRoot(Directory.GetCurrentDirectory())
                .UseIISIntegration()
                .UseStartup<Startup>()
                .Build();

basically does the reverse proxy magic when fronting your app with IIS.

Hi Brock,
Is the IISIntegration extension intended to handle reverse proxies generally, or is it specific to IIS? Looking at the source, it seems the latter, but I'm sure I could well be missing something.

I don't know to be honest -- so much changed in that area the months before RTM, and I've not had a chance to re-research it. But when I am using it locally with IIS Express it seems to do some magic to make requests look as if they're from the outside (meaning the host name is corrected, etc.).

There is a more generic "reverse proxy headers" middleware - forgot the name. But this is seems to be the recommendation for non-IIS scenarios.

I think it is this one:
https://github.com/aspnet/BasicMiddleware/tree/dev/src/Microsoft.AspNetCore.HttpOverrides
closing - feel free to open a new issue or report back if that works for you.

I have the same problem. I'm using the HttpOverrides to forward the host (X-Forwarded-Host), but it seems that IdentityServer is ignoring it, because the ./well-known/openid-configuration uses the internal ip and port instead of the forwarded one.
Is there maybe something to be configured on the IdentityServer application to consider those values?
I'm using nginx as reverse proxy.

This is unrelated to identityserver - it is asp.net that must make the translation before calling us.
Put a dummy middleware behind the overrides one and inspect the request. Maybe nginx does not send the right headers.

I also use HttpOverrides and it works correctly for me on self host IdS with nginx in front. I forward X-Forwarded-Host and X-Forwarded-Proto. Maybe problem is in that where you registered HttpOverrides, this middleware must be before UseIdentityServer, because then is registered BaseUrlMiddleware which store host name in the HttpContext.

@sbebrys Can you share some sample (from your code snippet) just to be able to configure here as well ? I have tried this nuget package (HttpOverrides) and have registered before any identityserver use (app.use).
I have my reverse proxy (apache) redirecting /.well-known and /connect endpoints to my docker container running aspnet.core and identityserver.
And when I access https://URL/.well-known/openid-configuration: Issuer is okay (external url) but all the rest is using internal urls

So I just figured this out for using nginx as a reverse proxy.
Be sure your nginx configuration looks something like the following:
server {
    listen 80;
    location / {
        proxy_pass http://localhost:5000;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection keep-alive;
        proxy_set_header Host $host;
        proxy_cache_bypass $http_upgrade;
    }
}

additionally add the Microsoft.AspNetCore.HttpOverrides middleware to your application and use it like this (add it before app.UseIdentityServer();):
app.UseForwardedHeaders(new ForwardedHeadersOptions
            {
                ForwardedHeaders = ForwardedHeaders.XForwardedHost | ForwardedHeaders.XForwardedProto
            });

@leastprivilege That's not a solution so it shouldn't be closed.

I just wanted to alternative solution to this one.
Nginx config:
location / {
	proxy_http_version        1.1;
	proxy_pass                http://identityserver;
	proxy_set_header Host     $host;
}

Startup.cs:
app.Use((context, next) =>
{
	if (Environment.GetEnvironmentVariable("SSL_OFFLOAD") == "true")
		context.Request.Scheme = "https";

	return next();
});

app.UseIdentityServer();

That why you don't need HttpOverrides middleware at all. IdentityServer will read hostname from "Host" header and on environments where you have SSL offload in use you just add environment variable SSL_OFFLOAD=true
@megamindbrian Hopefully this will help with your issue too.

@cblaettl, I used your recomendation, but I get the result without the /api path

Did you also get this behavior?

For me, this was the key in the latest version (2.2):
services.AddIdentityServer(options =>
      {
        options.IssuerUri = Configuration["jwt:authority"];
        options.PublicOrigin = options.IssuerUri;
      });

The PublicOrigin setting is required to change the root URL everywhere (specifically in discovery for bearer tokens). This allows me to host my IdentityServer4 server on the root but then register it somewhere else on my reverse proxy like /openid.

Hi,
here is the experience that I have made.
Let consider the method:
        private void ConfigureHttpsForwardingBehindProxy(IApplicationBuilder app)
        {
            var fordwardedHeaderOptions = new ForwardedHeadersOptions
            {
                ForwardedHeaders = ForwardedHeaders.XForwardedFor | ForwardedHeaders.XForwardedProto
            };
            fordwardedHeaderOptions.KnownNetworks.Clear();
            fordwardedHeaderOptions.KnownProxies.Clear();

            app.UseForwardedHeaders(fordwardedHeaderOptions);
        }

If you do
...
        app.UseIdentityServer();

        ConfigureHttpsForwardingBehindProxy(app);

...

Then you will get http but if you do this way
            ConfigureHttpsForwardingBehindProxy(app);

            app.UseIdentityServer();

Then you become https.
My conclusion is:
1 you need the nginx configuration with minimum
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;

2 and this is the most important, the order of applying the code is important


Cannot exit visual studio

VSCode Version: 1.4.0
OS Version: 10.11.6 (15G31)

Steps to Reproduce:

Open Terminal
run code .
quit visual studio
dock still shows icon and cannot force quit, it even blocks restart because process cannot be killed but there are no more processes named code or code helper in the Activity Monitor

I am very sorry to hear you are having issues with VS Code. I wonder if this issue is being caused by an installed extension. Can you try to run VS Code without extensions? From the command line, execute: code --disable-extensions and try your steps again to see if it reproduces. If you see it is an issue with the extension, please file it against the extension repository itself.

@bpasero - thank you for your response.

I tried from terminal with: code --disable-extensions
I exited with Code > Quit Visual Studio Code
But I still have the same problem

Can we try with our insider release?
We are releasing preview releases of the next stable VS Code version for everyone to try and give feedback. These preview releases are not 100% tested and might be unstable but contain our latest features and bugfixes. You can give our preview releases a try from: http://code.visualstudio.com/Download#insiders

A number of days has past without answer on the questions asked. I am thereby going to close this issue to clean up our backlog of issues. Once you can answer on the questions, the issue can be reopened again. Thanks for understanding this process!


maybe something wrong

sudo ./devweb
Password:
setupToken returned: Couldn't setup state  501: All the given peers are not reachable (Tried to connect to each peer twice and failed) [0]
<IP_ADDRESS> - - [02/Jun/2015:12:04:57 +0800] "PUT /new HTTP/1.1" 400 25
curl --verbose -X PUT localhost:8087/new

Hostname was NOT found in DNS cache
Trying ::1...
Connected to localhost (::1) port 8087 (#0)

PUT /new HTTP/1.1
User-Agent: curl/7.37.1
Host: localhost:8087
Accept: /

< HTTP/1.1 400 Bad Request
< Content-Length: 25
< Content-Type: text/plain; charset=utf-8
< Date: Tue, 02 Jun 2015 04:04:58 GMT
<
Unable to generate token

Connection #0 to host localhost left intact

in #43 you can now specify the url of your etcd cluster to connect to with --etcd, which should resolve this problem.


Added optional parameters getDataFn and getDataFnContext

Hello @begriffs,
I had to stick to angular services to reach our API endpoints in a large project (a lot of tweaks here are done at .config() time to adjust for dev/stage/production environments).
I could successfully stick to angular-paginate-anything :) with the small fix in this PR, which also addresses #42 by @stephaneeybert. Can you please review it? If it's something you'd like to merge, I'd be happy to make it more generic and documented. Let me know!
Best Regards,
Nuba
BTW this is how I use it in my template (jade)
   bgf-pagination(
     collection="vm.manyThings"
     url="vm.paginateAnythingParams.url"
     url-params="vm.paginateAnythingParams.urlParams"
     get-data-fn="vm.paginateAnythingGetData"
     get-data-fn-context="vm"
   )

and in my code (typescript)
    public paginateAnythingGetData(httpReqConfig) {
      // this is what angular-paginate-anything-does
      // return $http(httpReqConfig);

      // myApiService.apiMethod returns $http(...) too, so we're good
      return this.myApiService.apiMethod(
          httpReqConfig.params.foo,
          httpReqConfig.params.bar,
          httpReqConfig.headers['Range'],
          httpReqConfig.headers['Range-Unit'],
          this.cacheKey
    );

Looks like a good idea to make the library more flexible. If you provide good docs for the feature and ensure it's backwards compatible (i.e. omitting get-data-fn acts the same), and add a test then it should be good to add!
Thanks for taking the time to contribute back to the project.

(bump)

I'll look into it this week! :)
On Mon, Dec 7, 2015 at 9:59 PM, Joe Nelson<EMAIL_ADDRESS>wrote:

(bump)
—
Reply to this email directly or view it on GitHub
https://github.com/begriffs/angular-paginate-anything/pull/87#issuecomment-162709669
.

Hey the master branch test suite has been fixed! If you're still interested in sending this PR please rebase off the new master.

 Feel free to open it again if you'd like to complete the documentation.


Release v0.4.1

We've had a couple of new (non-breaking) feature additions and our docs are now out-of-sync with PyPI, so I'm going to cut a patch release!

[x] Remove dev0 from the version in __init__.py __version__

[x] Make a release commit: git commit -m 'RLS: v0.2.0'

[x] Push the RLS commit git push upstream master

[x] Make a GitHub release

[x] Call the release the current version, e.g. v0.2.0 and pin it to master
[x] This will trigger this github action job. Ensure that it completes in the actions tab
After a few minutes, confirm that the PyPI version is updated
[x] If it's a minor/major versoin bump, add some simple release notes.
If you wish, use [github-activity to generate a changelog](https://github.com/choldgraf/github-activity, eg github-activity pandas-dev/pydata-sphinx-theme --since v0.2.2 --until v0.3.0).

[x] Bump the __init__.py __version__ and add dev0 to it. e.g.: v0.2.1dev0.

[x] Commit and push to master
```
git commit -m "dev version bump"
git push upstream master
```

[x] Celebrate, you're done!

now released! https://pypi.org/project/pydata-sphinx-theme/

Thanks for the release!


Add comand-line mode functionality

Adds auto-closing, auto-deleting and auto-escaping in command-line mode. Any pair can be disabled in command-line mode using a new key disable_command_mode.

Thanks for the great idea! However some part of the code can be better. The command_get_pair() and insert_get_pair() have many repeated code. You can add a new parameter to the get_pair() and do different thing based on the value instead of writing 2 different functions.


luci-app-attendedsysupgrade: update status keys and messages

Maintainer: @aparcar
Tested on: x86/64 SNAPSHOT build
The reported status values have changed on the ASU server several times since the 'steps' list has been updated, so update it to include all possible values.  Although some of the values in the list are no longer presented by the official OpenWrt ASU server, they may still be produced by other ASU servers, so leave them in for now.
Percentages have been adjusted based upon measurement of the build phases for several build configurations (ath79 vs x86, few packages vs many...).  The numbers are still (very) rough estimates, but are more representative than the old values.

LGTM. I assume we should wait for https://github.com/openwrt/asu/pull/1037 to be merged and deployed to sysupgrade.openwrt.org? Otherwise we'll break the LuCI app on SNAPSHOT.

@dannil , thanks for looking at this.
I dug back through the ASU server revisions a couple months and couldn't find where those old status values were dropped, so we've been running without them for a while.  But, I know ImmortalWrt is running an older version ASU server, so it might still be producing them and I wanted to let their server still work when they upgrade their LuCI app...

Should I merge this?

@dannil , thanks for looking at this.
I dug back through the ASU server revisions a couple months and couldn't find where those old status values were dropped, so we've been running without them for a while. But, I know ImmortalWrt is running an older version ASU server, so it might still be producing them and I wanted to let their server still work when they upgrade their LuCI app...

If they're dropped it should be fine, the only weirdness I could see was that with the new UI the unpack_imagebuilder status has the same progress percentage as the newly modified calculate_packages_hash, which meant that the progress bar would stay still moving from unpack_imagebuilder to calculate_packages_hash if you ran the new LuCI app and the old ASU server, but if they're already removed from ASU anyway before this then it's not a problem.


disambiguate terms

I found that the same term was being used in the headers of different file output in slightly different forms. I'd like the same form used for each different header to remove confusion. Here are the examples that I found:
CountDecoy
CountTarget
Cumulative Decoy
Cumulative Target
cumulative_decoy
cumulative_target
matched ion counts
matched ions
Protein Full Name
Protein FullName
Sequence coverage
Sequence coverage %

Executive decision: Human readable, space-separated, individual words capitalized.

Once this is done, we'll have to update the glossary of terminology in the wiki

For psms tsv files, this is done in https://github.com/smith-chem-wisc/MetaMorpheus/pull/376
Still need to finish this for protein files.

Done in https://github.com/smith-chem-wisc/MetaMorpheus/pull/507


Do you have a getClasses function for makeStyle hook?

Hi,
I would like to get access to the component's classes into jest tests. In the previous versions, we were using HOC withStyles with your getClasses function inside tests.
Now, we are forced to use your hook makeStyles. How can I get access to my component's classes inside the jest tests?
Thanks!

@MaximKudriavtsev Maybe you can export the useStyles() hook? But anyway, given the upcoming migration to styled-components, I would encourage you to defer this matter to react-jss.

I am getting below error, even after exporting export const useStyles = makeStyles(theme => ({.... from component file.
TypeError: useStyles is not a function

      13 |   mount = createMount({ strict: true });
      14 |   shallow = createShallow({ dive: true });
    >15 |   classes = getClasses(<StatusLabel content="Test" />);
         |                        ^
      16 | });

After seeing implementation in<EMAIL_ADDRESS>got to know it is expecting element.type.useStyles to be defined, which is not happening, @oliviertassinari do we need to specify this in some other way to get our unit tests working after converting our components from HOC withStyle to HOOK useStyles?

@hitendramalviya Given that we are progressively moving to styled-components, I would recommend stopping using @material-ui/styles. Have a look at react-jss if you want to keep using the concept (our styling solution is based on JSS).
In your case, you can do the following in the source:
const useStyles = makeStyles({});

function StatusLabel(props) {}

StatusLabel.useStyles = useStyles

This might give some inspiration too: https://github.com/mui-org/material-ui/pull/15023/files#diff-110ef9c4cc1990d293d5cc9911955aae.


Make ci-ipsec-upgrade a part of /test

As #27276 was fixed, it's time to make ci-ipsec-upgrade a part of /test.
Signed-off-by: Zhichuan Liang<EMAIL_ADDRESS>

/test


Fix ManagedSeed bootstrapping for Kubernetes 1.24+ clusters

How to categorize this PR?

/area usability
/kind bug
What this PR does / why we need it:
The non-default ServiceAccount bootstrapping mode is currently broken for 1.24+ clusters because it relies on the ServiceAccount controller to populate a static token secret. However, this is no longer happening automatically for 1.24+ clusters.
This PR adapts the code to use the serviceaccounts/token subresource instead.
Which issue(s) this PR fixes:
Fixes #7311
Release note:

A bug preventing `ManagedSeed` bootstrapping for 1.24+ Kubernetes clusters has been fixed.

/test pull-gardener-e2e-kind-ha-multi-zone


"drop measurement" hangs

Bug report
System info: Ubuntu 14.04, Kernel 3.16.0-46, influxdb 1.2.2-1 (from official repo)
Steps to reproduce:

"drop measurement process_virtual_memory_bytes"
wait
wait more
file bug report while still waiting (>2h so far)

Expected behavior: measurement to be dropped
Actual behavior: nothing... still waiting
Total influxdb database is not that large (~100GB), this series is probably not even 1% of that so this seems like a relatively tiny amount of data.
I've seen very similar issues with "drop series from process_virtual_memory_bytes" and "drop series where ...".
"debug/pprof/all?cpu=true" returns a 404 "Unknown profile: all" ([http] pprof-enabled = true in influxdb.conf)
vars: https://gist.github.com/AdamJacobMuller/bb44181c09f8e72c2b189b7293ed27c1
I know this isn't enough information to fix, but, I'm more than open to gathering more information, just let me know what you need. In particular, any ideas how to get pprof data?

influxdb eventually started rejecting writes, in an increasing percentage until:

[I] 2017-06-17T06:01:59Z write failed for shard 49: engine: cache-max-memory-size exceeded:<PHONE_NUMBER>/1048576000) service=write

a lot of these appeared, then memory usage eventually started to balloon until influxdb consumed all the memory in the system (24G) and the system OOM-killed influxdb. its doing whatever it does on crash recovery for ~15m now.

after recovery and an upgrade to 1.2.4:

drop measurement process_virtual_memory_bytes;
ERR: measurement not found: process_virtual_memory_bytes

after a few more crashes, I left the system in a state of not accepting any writes for ~12 hours, it did a lot of compaction and then I was able to run drop measurement in a normal timeframe (a few seconds per measurement), restored writes and everything seems fine, besides ~20 hours of missing data

also relevant, after restoring writes, I can confirm that there are no more writes to the measurements I was attempting to drop.

This should be fixed in 1.3. See #8204.

Possibly, this issue is still there.
Today, after I updated from 1.2.4 to 1.3.2 and restarted InfluxDB service, things were going smooth. Then, I installed Chronograf on the same instance.
To make some graph, I noticed the measurement list is extremely slow to load.
I immediately stopped the chronograph service. Then, started to cleanup some old and unnecessary measurements. First, couple ones were done almost instantly. They something went haywire.
Now, it is completed after ~1 hour, consuming 1.5GB+ memory.
Here's the output from show stats: https://gist.github.com/kmonsoor/cba814077959bdd237e7ebb3b968ad2b
Here's current htop screenshot. Notice the top item.

@kmonsoor Deletes will cause compactions to run to evict the deleted series data from TSM files.  Deletes statements could hang if one was running and was not at an interruptible spot.  #8629 fixes that.


Convert from phusion to debian + nosh

See http://homepage.ntlworld.com/jonathan.deboynepollard/Softwares/nosh.html

went with less for now


Several fixes in http client

Codecov Report

Merging #301 into release will increase coverage by 0.01%.
The diff coverage is 80.95%.

@@             Coverage Diff             @@
##           release     #301      +/-   ##
===========================================
+ Coverage    86.09%   86.11%   +0.01%
===========================================
  Files          132      132
  Lines        15074    15081       +7
===========================================
+ Hits         12978    12987       +9
+ Misses        2096     2094       -2

Impacted Files
Coverage Δ

pulsar/apps/greenio/utils.py
84% <ø> (ø)
:arrow_up:

pulsar/apps/test/__init__.py
93.75% <ø> (ø)
:arrow_up:

pulsar/apps/wsgi/wrappers.py
74.87% <0%> (-1.51%)
:arrow_down:

pulsar/apps/http/client.py
90.29% <100%> (+0.29%)
:arrow_up:

pulsar/apps/test/runner.py
80.23% <100%> (+0.11%)
:arrow_up:

pulsar/apps/test/utils.py
94.23% <100%> (+0.29%)
:arrow_up:

pulsar/async/concurrency.py
81.43% <0%> (+0.75%)
:arrow_up:

pulsar/utils/pylib/events.py
99.02% <0%> (+1.94%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f10c3ea...01183d7. Read the comment docs.


IndexError: list index out of range

Hello there,
I have ran into an issue where it is not running. After running the file itself I get the following:
Traceback (most recent call last):   File "./vgpu_unlock", line 246, in <module>     main()   File "./vgpu_unlock", line 230, in main     if sys.argv[1] != "-f": IndexError: list index out of range
I don't know if that is the issue for it not wanting to run, but that's what I suspect, as that's what I get while trying to run it. I did have to modify a few things in the code, including changing the path of where python3 was, but this is an issue I have no clue on what to do, and I couldn't find anything about it.
Thank you,
Austin Zechar

Update: found out that was normal when running the file without the driver file. I now have a new issue:
The systemctl file when run now displays this:
Nov 18 04:32:43 BillyBoy nvidia-vgpud[3151]: vGPU types: 401 Nov 18 04:32:43 BillyBoy nvidia-vgpud[3151]:  Nov 18 04:32:43 BillyBoy nvidia-vgpud[3165]: vgpu_unlock loaded. Nov 18 04:32:43 BillyBoy modprobe[3165]: vgpu_unlock loaded. Nov 18 04:32:43 BillyBoy nvidia-vgpud[3151]: error: failed to allocate client: 59 Nov 18 04:32:43 BillyBoy nvidia-vgpud[3151]: error: failed to read pGPU information: 9 Nov 18 04:32:43 BillyBoy nvidia-vgpud[3151]: error: failed to send vGPU configuration info to RM: 9 Nov 18 04:32:43 BillyBoy nvidia-vgpud[3151]: PID file unlocked. Nov 18 04:32:43 BillyBoy nvidia-vgpud[3151]: PID file closed. Nov 18 04:32:43 BillyBoy nvidia-vgpud[3151]: Shutdown (3151)

hello?

@W1ldAustin Hello, tried to unlock my gpu today and ended with this same problem (failed to allocate client 59, etc etc). Have you managed to fix it somehow? Thanks.


Provide move file function in Project Data tab

Now that we have the ability to create directories in the Project Data tab, the next thing the user will want to do is to move files into it using drag and drop.
Will first need API support.

Suggest the following: -

/file/move endpoint to move unmanaged files (to a new path)
/path/move endpoint to move paths

I worry that any path that contains a ProjectFile/Managed File (an instance of a Dataset in a project) will cause all sorts of problems in the backend. So I think moving and deleting paths that contain ProjectFile instances will need to result in one of two actions: -

Update the corresponding ProjectFile records to reflect the new path
Prevent the move/removal


Removing movieclip inside onComplete callback

A bug looks like https://github.com/GoodBoyDigital/pixi.js/issues/235, but not sure because I dont know coffeescript.
When I write something like this
stage.addChild(movieclip);
movieclip.onComplete = function(){ stage.removeChild(this) }

I can get an error Cannot read property 'updateTransform' of undefined
The problem is caused by changing length of stage.children in this cycle
PIXI.Stage.prototype.updateTransform = function() {
    this.worldAlpha = 1;
    for(var i=0,j=this.children.length; i<j; i++) {
        this.children[i].updateTransform();
    }
...

When PIXI calls updateTransform() of my "completed" movieclip, movieclip.onComplete() fires immediately and changes stage.children array. To close the issue, you should rethink this algorithm.
For now I use this workaround, just to pull out array changing from cycle.
movieclip.onComplete = function() {
    setTimeout(function(t) {
        stage.removeChild(t);
    }, 15, this);
};

This should no longer be an issue in v3. cheers!


Mic not automatically recognized on laptop

Hello,
I'm trying the code on my linux laptop (ubuntu-based, Linux mint 20). The installation went well but when arriving to "Say something" there are some error messages related to the microphone detection:
ALSA lib pcm.c:2642:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.rear
ALSA lib pcm.c:2642:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.center_lfe
ALSA lib pcm.c:2642:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.side
ALSA lib pcm_route.c:869:(find_matching_chmap) Found no matching channel map
ALSA lib pcm_route.c:869:(find_matching_chmap) Found no matching channel map
ALSA lib pcm_route.c:869:(find_matching_chmap) Found no matching channel map
ALSA lib pcm_route.c:869:(find_matching_chmap) Found no matching channel map
ALSA lib pcm_oss.c:377:(_snd_pcm_oss_open) Unknown field port
ALSA lib pcm_oss.c:377:(_snd_pcm_oss_open) Unknown field port
ALSA lib pcm_usb_stream.c:486:(_snd_pcm_usb_stream_open) Invalid type for card
ALSA lib pcm_usb_stream.c:486:(_snd_pcm_usb_stream_open) Invalid type for card

I've installed the additional portions as mentioned on the README but still no dice.
How would we get this sorted?
Thanks!

To solve this situation I've attached a USB microphone and then worked great, albeit still with the same error message.
Just seems unable to get it directly from the laptop microphone. Other than that, excellent work.

Yes, the message occurs even when working.  I was not aware built-in mics would not work. Perhaps the threshold just needs to be lowered?

I've tested with headphone mics, a dedicated microphone and finally a conference room mic/speaker.
The conference room one works best for this purpose, but no internal mic yet on Linux (Mint 20). Anyways, not a big deal.

Unfortunately it fails to detect built-in mic on Ubuntu 22.04 Mate.
        python mic.py --model tiny
        ALSA lib pcm_dsnoop.c:601:(snd_pcm_dsnoop_open) unable to open slave
        ALSA lib pcm_dmix.c:1032:(snd_pcm_dmix_open) unable to open slave
        ALSA lib pcm.c:2664:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.rear
        ALSA lib pcm.c:2664:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.center_lfe
        ALSA lib pcm.c:2664:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.side
        ALSA lib confmisc.c:160:(snd_config_get_card) Invalid field card
        ALSA lib pcm_usb_stream.c:482:(_snd_pcm_usb_stream_open) Invalid card 'card'
        ALSA lib confmisc.c:160:(snd_config_get_card) Invalid field card
        ALSA lib pcm_usb_stream.c:482:(_snd_pcm_usb_stream_open) Invalid card 'card'
        ALSA lib pcm_dmix.c:1032:(snd_pcm_dmix_open) unable to open slave
        Say something!

There is no response.


Fix issue #352

Coverage remained the same at 100.0% when pulling eba6d0f6ecd2c90b612eb599a86498617be56835 on issue_352 into b17641f176c490be2d19c26a02766265abd8894c on master.


fix(sec): upgrade org.springframework:spring-context to 5.3.19

What happened？
There are 1 security vulnerabilities found in org.springframework:spring-context 2.5.6.SEC03

CVE-2022-22968

What did I do？
Upgrade org.springframework:spring-context from 2.5.6.SEC03 to 5.3.19 for vulnerability fix
What did you expect to happen？
Ideally, no insecure libs should be used.
The specification of the pull request
PR Specification from OSCS

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


pyopenvdb file streaming

Is your feature request related to a problem? Please describe.
Currently pyopenvdb only supports writing to local disk. This prevents writing directly to a destination in some cases (e.g. to cloud storage).
Describe the solution you'd like
The vdb read and write functions accept a file object (or pathlike object) instead of a str.
Describe alternatives you've considered
Currently our workaround is to write to a temporary file on local disk and copy it's contents to the destination.

Here's a recent example of OpenCV modifying their C++ library to accept pathlib objects, maybe it would be helpful:
https://github.com/opencv/opencv/pull/24773/files

Though this is not currently a priority for the Steering Committee, we agree that this would be a worthwhile improvement. Note that we've recently adopted pybind11 as our python binding library. Feel free to make a PR if it's something you think you might have time to contribute to OpenVDB. Thanks for the suggestion.


Feature/escalation

Current coverage is 87.61% (diff: 29.03%)

No coverage report found for master at 83a4124.

Powered by Codecov. Last update 83a4124...fddb07d


Can we only subscribe to collection event data?

Hey Guys,
This repo is amazing by far. Very happy you guys made this. Only problem is , I don't see any options to subscribe to wallet addresses events?
If their any wya we can subscribe to wallet address else besides collections?

Hey @ceobitch, Currently we are in beta and currently support the following event types on a per-collection basis:
item listed
item sold
item transferred
item metadata updates
item canceled
item received offer
item received bid
I've noted your request

options to subscribe to wallet addresses events

and will share it with the team.


stackdriver: update changelog before release

I published  6245c6350fab3bb5815c60f8479688980c885277 as opentelemetry-stackdriver 0.20.0. If someone wants to add a tag, that would be great.

I published 6245c63 as opentelemetry-stackdriver 0.20.0. If someone wants to add a tag, that would be great.

Thanks @djc. Have created the tag on this commit-id.


Wordpress Elementor Plugin Error 403 on Save

Elementor Plugin
I am getting a 403 error when I try to save some changes that I made to my page using the html/text editor.
Server error Log:
"POST /wp-admin/admin-ajax.php HTTP/1.0" 200 943 "https://xxxxxxxxxx/wp-admin/post.php?post=9&action=elementor"
I will update this post if I find any more information.

Dear @canyon1991 , your log entry appears to have a 200 (OK) status code. Is it possible you pasted the wrong line? In any case, we would need the full ModSecurity audit log entry in order to help you.

Like a dummy I posted the wrong line.
Server Log Entry:
"POST /wp-admin/admin-ajax.php HTTP/1.0" 403 3321 "https://xxxxxxxx/wp-admin/post.php?post=9&action=elementor"
From the Audit Log:
Message: Warning. detected XSS using libinjection. [file "/etc/modsecurity/coreruleset/rules/REQUEST-941-APPLICATION-ATTACK-XSS.conf"] [line "98"] [id "941100"] [msg "XSS Attack Detected via libinjection"] [data "Matched Data: XSS data found within ARGS:actions: {\x22save_builder\x22:{\x22action\x22:\x22save_builder\x22,\x22data\x22:{\x22status\x22:\x22publish\x22,\x22elements\x22:[{\x22id\x22:\x226b339b9\x22,\x22elType\x22:\x22container\x22,\x22isInner\x22:false,\x22isLocked\x22:false,\x22settings\x22:{},\x22elements\x22:[{\x22id\x22:\x2238fc251\x22,\x22elType\x22:\x22widget\x22,\x22isInner\x22:false,\x22isLocked\x22:false,\x22settings\x22:{\x22editor\x22:\x22C 2023 T2G Des..."] [severity "CRITICAL"] [ver "OWASP_CRS/4.0.0-rc2"] [tag "application-multi"] [tag "language-multi"] [tag "platform-multi"] [tag "attack-xss"] [tag "xss-perf-disable"] [tag "paranoia-level/1"] [tag "OWASP_CRS"] [tag "capec/1000/152/242"]
Message: Warning. Pattern match "(?i)<[^0-9<>A-Z_a-z](?:[^\s\v"'<>]:)?[^0-9<>A-Z_a-z][^0-9A-Z_a-z]?(?:s[^0-9A-Z_a-z]?(?:c[^0-9A-Z_a-z]?r[^0-9A-Z_a-z]?i[^0-9A-Z_a-z]?p[^0-9A-Z_a-z]?t|t[^0-9A-Z_a-z]?y[^0-9A-Z_a-z]?l[^0-9A-Z_a-z]?e|v[^0-9A-Z_a-z]?g|e[^0-9A-Z_a-z]?t[^0-9> ..." at ARGS:actions. [file "/etc/modsecurity/coreruleset/rules/REQUEST-941-APPLICATION-ATTACK-XSS.conf"] [line "217"] [id "941160"] [msg "NoScript XSS InjectionChecker: HTML Injection"] [data "Matched Data: \x22save_builder\x22:{\x22action\x22:\x22save_builder\x22,\x22data\x22:{\x22status\x22:\x22publish\x22,\x22elements\x22:[{\x22id\x22:\x226b339b9\x22,\x22elType\x22:\x22container\x22,\x22isInner\x22:false,\x22isLocked\x22:false,\x22settings\x22:{},\x22elements\x22:[{\x22id\x22:\x2238fc251\x22,\x22elType\x22:\x22widget\x22,\x22isInner\x22:false,\x22isLocked\x22:false,\x22settings\x22:{\x22editor\x22:\x22C 2023 T2G Design Ltd | Registered in England No.13..."] [severity "CRITICAL"] [ver "OWASP_CRS
Message: Access denied with code 403 (phase 2). Operator GE matched 5 at TX:blocking_inbound_anomaly_score. [file "/etc/modsecurity/coreruleset/rules/REQUEST-949-BLOCKING-EVALUATION.conf"] [line "186"] [id "949110"] [msg "Inbound Anomaly Score Exceeded (Total Score: 10)"] [ver "OWASP_CRS/4.0.0-rc2"] [tag "anomaly-evaluation"]
Message: Warning. Unconditional match in SecAction. [file "/etc/modsecurity/coreruleset/rules/RESPONSE-980-CORRELATION.conf"] [line "96"] [id "980170"] [msg "Anomaly Scores: (Inbound Scores: blocking=10, detection=10, per_pl=10-0-0-0, threshold=5) - (Outbound Scores: blocking=0, detection=0, per_pl=0-0-0-0, threshold=4) - (SQLI=0, XSS=10, RFI=0, LFI=0, RCE=0, PHPI=0, HTTP=0, SESS=0, COMBINED_SCORE=10)"] [ver "OWASP_CRS/4.0.0-rc2"] [tag "reporting"]
Apache-Error: [file "apache2_util.c"] [line 275] [level 3] [client xxx.xx.xxx.xx] ModSecurity: Warning. detected XSS using libinjection. [file "/etc/modsecurity/coreruleset/rules/REQUEST-941-APPLICATION-ATTACK-XSS.conf"] [line "98"] [id "941100"] [msg "XSS Attack Detected via libinjection"] [data "Matched Data: XSS data found within ARGS:actions: {\\x22save_builder\\x22:{\\x22action\\x22:\\x22save_builder\\x22,\\x22data\\x22:{\\x22status\\x22:\\x22publish\\x22,\\x22elements\\x22:[{\\x22id\\x22:\\x226b339b9\\x22,\\x22elType\\x22:\\x22container\\x22,\\x22isInner\\x22:false,\\x22isLocked\\x22:false,\\x22settings\\x22:{},\\x22elements\\x22:[{\\x22id\\x22:\\x2238fc251\\x22,\\x22elType\\x22:\\x22widget\\x22,\\x22isInner\\x22:false,\\x22isLocked\\x22:false,\\x22settings\\x22:{\\x22editor\\x22:\\x22C 2023 T2G Des..."] [severity "CRITICAL"] [ver "OWASP_CRS/4.0.0-rc2"] [tag "application-multi"] [tag "language-multi"] [tag "platform-multi"] [tag "attack-xss"] [tag "xss-perf-disable"] [tag "paranoia-level/1"] [tag "OWASP_CRS"] [tag "capec/1000/152/242"] [hostname "sandbox.mywebsite.com"] [uri "/wp-admin/admin-ajax.php"] [unique_id "ZVJvFvWBfL52WMRYD0i2DwAAAA4"]
Apache-Error: [file "apache2_util.c"] [line 275] [level 3] [client xxx.xx.xxx.xx] ModSecurity: Warning. Pattern match "(?i)<[^0-9<>A-Z_a-z](?:[^\\\\s\\\\v\\"'<>]:)?[^0-9<>A-Z_a-z][^0-9A-Z_a-z]?(?:s[^0-9A-Z_a-z]?(?:c[^0-9A-Z_a-z]?r[^0-9A-Z_a-z]?i[^0-9A-Z_a-z]?p[^0-9A-Z_a-z]?t|t[^0-9A-Z_a-z]?y[^0-9A-Z_a-z]?l[^0-9A-Z_a-z]?e|v[^0-9A-Z_a-z]?g|e[^0-9A-Z_a-z]?t[^0-9> ..." at ARGS:actions. [file "/etc/modsecurity/coreruleset/rules/REQUEST-941-APPLICATION-ATTACK-XSS.conf"] [line "217"] [id "941160"] [msg "NoScript XSS InjectionChecker: HTML Injection"] [data "Matched Data: \\x22save_builder\\x22:{\\x22action\\x22:\\x22save_builder\\x22,\\x22data\\x22:{\\x22status\\x22:\\x22publish\\x22,\\x22elements\\x22:[{\\x22id\\x22:\\x226b339b9\\x22,\\x22elType\\x22:\\x22container\\x22,\\x22isInner\\x22:false,\\x22isLocked\\x22:false,\\x22settings\\x22:{},\\x22elements\\x22:[{\\x22id\\x22:\\x2238fc251\\x22,\\x22elType\\x22:\\x22widget\\x22,\\x22isInner\\x22:false,\\x22isLocked\\x22:false,\\x22settings\\x22:{\\x22editor\\x22:\\x22C 2023 T2G Design Ltd | Registered in England No.13..."] [severity "CRITICAL"] [ver "OWASP_CRS [hostname "sandbox.mywebsite.com"] [uri "/wp-admin/admin-ajax.php"] [unique_id "ZVJvFvWBfL52WMRYD0i2DwAAAA4"]
Apache-Error: [file "apache2_util.c"] [line 275] [level 3] [client xxx.xx.xxx.xx] ModSecurity: Access denied with code 403 (phase 2). Operator GE matched 5 at TX:blocking_inbound_anomaly_score. [file "/etc/modsecurity/coreruleset/rules/REQUEST-949-BLOCKING-EVALUATION.conf"] [line "186"] [id "949110"] [msg "Inbound Anomaly Score Exceeded (Total Score: 10)"] [ver "OWASP_CRS/4.0.0-rc2"] [tag "anomaly-evaluation"] [hostname "sandbox.mywebsite.com"] [uri "/wp-admin/admin-ajax.php"] [unique_id "ZVJvFvWBfL52WMRYD0i2DwAAAA4"]
Apache-Error: [file "apache2_util.c"] [line 275] [level 3] [client xxx.xx.xxx.xx] ModSecurity: Warning. Unconditional match in SecAction. [file "/etc/modsecurity/coreruleset/rules/RESPONSE-980-CORRELATION.conf"] [line "96"] [id "980170"] [msg "Anomaly Scores: (Inbound Scores: blocking=10, detection=10, per_pl=10-0-0-0, threshold=5) - (Outbound Scores: blocking=0, detection=0, per_pl=0-0-0-0, threshold=4) - (SQLI=0, XSS=10, RFI=0, LFI=0, RCE=0, PHPI=0, HTTP=0, SESS=0, COMBINED_SCORE=10)"] [ver "OWASP_CRS/4.0.0-rc2"] [tag "reporting"] [hostname "sandbox.mywebsite.com"] [uri "/error/403.html"] [unique_id "ZVJvFvWBfL52WMRYD0i2DwAAAA4"]
Action: Intercepted (phase 2)
Stopwatch:<PHONE_NUMBER>448869 27429 (- - -)
Stopwatch2:<PHONE_NUMBER>448869 27429; combined=26618, p1=1041, p2=25339, p3=0, p4=0, p5=238, sr=0, sw=0, l=0, gc=0
Producer: ModSecurity for Apache/2.9.7 (http://www.modsecurity.org/); OWASP_CRS/4.0.0-rc2.
Server: Apache/2.4.58 (Ubuntu) OpenSSL/3.0.2
Engine-Mode: "ENABLED"

I was able to get my pages to save once I commented out these 2 rules:
-=[ Libinjection - XSS Detection ]=- (starts at the line 66 of /etc/modsecurity/coreruleset/rules/REQUEST-941-APPLICATION-ATTACK-XSS.conf)
&
-=[ NoScript XSS Filters ]=- (starts at the line 188 of /etc/modsecurity/coreruleset/rules/REQUEST-941-APPLICATION-ATTACK-XSS.conf)
I am out of my depth of this subject, so I don't know if these 2 rules are critical but for the time being, I am keeping them disabled - I guess it is better than turning off all the ModSecurity Rules.
There errors are still showing up on the audit log - even the commented out rules, now I am even more confused? On other hand more Apache-Errors now though.
Message: Warning. Pattern match "(?i)(?:t["\^]i["\^]m["\^]e|[\n\r;`\{]|\|\|?|&&?)[\s\v][\s\v"'-\(,@](?:["'\.-9A-Z_a-z]+/|(?:["'\x5c\^][0-9A-Z_a-z]["'\x5c\^]:.|[ "'\.-9A-Z\x5c\^-_a-z])\x5c)?["\^](>
Message: Warning. detected XSS using libinjection. [file "/etc/modsecurity/coreruleset/rules/REQUEST-941-APPLICATION-ATTACK-XSS.conf"] [line "97"] [id "941100"] [msg "XSS Attack Detected via libinjection"] [data "Matched Data: XSS data f>
Message: Access denied with code 403 (phase 2). Operator GE matched 5 at TX:blocking_inbound_anomaly_score. [file "/etc/modsecurity/coreruleset/rules/REQUEST-949-BLOCKING-EVALUATION.conf"] [line "186"] [id "949110"] [msg "Inbound Anomaly>
Message: Warning. Unconditional match in SecAction. [file "/etc/modsecurity/coreruleset/rules/RESPONSE-980-CORRELATION.conf"] [line "96"] [id "980170"] [msg "Anomaly Scores: (Inbound Scores: blocking=5, detection=5, per_pl=5-0-0-0, thres>
Apache-Error: [file "apache2_util.c"] [line 275] [level 3] [client <IP_ADDRESS>] ModSecurity: Warning. Pattern match "(?i)(?:t[\\"\\\\^]*i[\\"\\\\^]*m[\\"\\\\^]*e|[\\\\n\\\\r;`\\\\{]|\\\\|\\\\|?|&&?>
Apache-Error: [file "apache2_util.c"] [line 275] [level 3] [client <IP_ADDRESS>] ModSecurity: Warning. detected XSS using libinjection. [file "/etc/modsecurity/coreruleset/rules/REQUEST-941-APPLICATION-ATTACK-XSS.conf"] [line "97"] [id >
Apache-Error: [file "apache2_util.c"] [line 275] [level 3] [client <IP_ADDRESS>] ModSecurity: Access denied with code 403 (phase 2). Operator GE matched 5 at TX:blocking_inbound_anomaly_score. [file "/etc/modsecurity/coreruleset/rules/R>
Apache-Error: [file "apache2_util.c"] [line 275] [level 3] [client <IP_ADDRESS>] ModSecurity: Warning. Unconditional match in SecAction. [file "/etc/modsecurity/coreruleset/rules/RESPONSE-980-CORRELATION.conf"] [line "96"] [id "980170"]>
At least I can save my files for now.

@canyon1991 Unfortunately, we don't support any WordPress plugins or themes. Anyway, try this exclusion rule:
SecRule REQUEST_FILENAME "@endsWith /wp-admin/post.php" \
    "id:9990399,\
    phase:2,\
    pass,\
    t:none,\
    nolog,\
    chain"
    SecRule ARGS:action "@streq elementor" \
        "t:none,\
        chain"
        SecRule &ARGS:action "@eq 1" \
            "t:none,\
            ctl:ruleRemoveTargetById=941100;ARGS:actions,\
            ctl:ruleRemoveTargetById=941160;ARGS:actions"

@azurit Thank you for sending this over and trying to help me and the anybody else that has the same issue.
I do understand your point of "support any WordPress plugins or themes", but I believed the whole point of the "wordpress-rule-exclusions-plugin" was to help people that use WordPress not have issues in general.  I understand that used issue is caused because of a plugin in my case, but surely there will be many other plugins in general that will have this issue too.
On a side note, on which file would I add this exclusion to?

The problem is just what you said - there are way too much plugins and it's really hard to support it. I'm running CRS with WordPress on my commercial services and we already have almost 400 exclusion rules for various WordPress plugins and themes, while new rules are added every week.
Put that rule into file REQUEST-900-EXCLUSION-RULES-BEFORE-CRS.conf.

@azurit I understand it now. This might be a silly question but can't rules be made broader to avoid having too many false positives or would we be creating a lot of security holes?
Are there any reliable resources I can use to learn how to create my own exclusions? Thanks to your exclusion, I was able to understand a little bit about it.

@canyon1991 Every rule was written to prevent a specific attack so it's probably not possible to make them less sensitive while still fully preventing a threat for which it was created.
Of course you can modify my exclusion rule and remove check for action parameter, so it will disable listed rules for every request to file /wp-admin/post.php but this will probably lower the CRS protection too much. Exclusion rules should be as tight as possible to not open more holes in the firewall as is needed.
Try looking at this.

@azurit I really appreciate your help with this.

@canyon1991 Exclusion rule is ok but you sent a different data in this request so it triggered also another rule - 932370. Try to modify my rule and exclude also ID 932370.

@azurit  I totally understand if you don't have too much time on your hands, but could you give me a brief idea/example how to read these things so in the future I can make my own exclusions without having to bother you and the other members of the community?
I really would like to learn, and I am happy to investigate things and make mistakes but if I know where to look, it makes things slightly easier.

@canyon1991 : I recommend you read our documentation and also read through the tutorials hosted at netnea.com. There is also a script (modsec-rulereport.rb) that allows you pipe rule alerts into and then received the finished rule exclusion. On top the script will refuse to exclude one of the delicate rules @azurit mentions.

Ps: does that mean I shouldn't be add any exceptions for the last 2 lines you sent as an example?

Yes, exactly.

@canyon1991 Were you able to write correct exclusion rule?

I haven't addressed it yet, but it's definitely on my radar to get done at some point once I have more time to invest on learning about MOD Security. For now, I am only using basic security measures that come out of the box with the services I use on top of extremely strict Cloudflare WAF rules.


Adding the incremental entity provider backend

Signed-off-by: Damon Kaswell<EMAIL_ADDRESS>Hey, I just made a Pull Request!
This PR introduces the concept of incremental entity providers to Backstage. An incremental entity provider is a form of entity provider that can ingest from very large data sources where a full mutation to achieve initial ingestion may be impossible.
Incremental providers split a large ingestion task into smaller chunks which can be streamed into the catalog (using deltas under the hood). The size and length of these chunks are configurable, as are the lengths of the pauses between them and the amount of time to rest after ingestion is complete before starting a fresh run.
The incremental entity provider tracks a cursor to indicate its current location in the data stream. This allows the provider's work to pick up where it leaves off after a burst of activity, on any replica running the provider's task.
In order to accomplish this, the incremental provider introduces a new schema called ingestion to the Postgres database, with three tables:

ingestions - Contains the status record for an incremental entity provider. Over time, there will always be two records per provider: One for the previous run, and one for the current run. Older ingestion records are purged automatically.
ingestion_marks - Contains the cursor records used by the incremental entity provider. They have a many-to-one relationship with the ingestions table records. Ingestion marks for old ingestion records are purged automatically as well.
ingestion_mark_entities - Contains an entry for each discovered entity, with a many-to-one relationship with records from the ingestion_marks table. Again, older records are purged automatically. This table is used for comparisons to the final_entities table, to allow entities for assets that have been removed to be deleted.

It's important to note that due to the fact that any replica of Backstage could pick up the task of ingestion, stateful protocols such as LDAP will not work, and should be ingested either via a proxy or with a standard entity provider.
:heavy_check_mark: Checklist

[x] A changeset describing the change and affected packages. (more info)
[ ] Added or updated documentation
[ ] Tests for new functionality and regression tests for bug fixes
[ ] Screenshots attached (for UI changes)
[x] All your commits have a Signed-off-by line in the message. (more info)

Whoa! Nice :) Review to be had later on. Regarding the DI error, note that all our in-repo dependencies (to other packages/plugins packages) are on the form "workspace:^" now, instead of actual versions. Way more convenient, I'm sure you'll find. I think that should let the compilation progress more.

Thanks for the heads-up! I also need to trim some other dependencies that are unneeded in this version.

Not stale

Just closing and opening to try to nudge the builds


Problems with Stretch properties

What's Wrong
When configuring the Stretch properties (display/window/stretch/mode and display/window/stretch/aspect), the prop images do not behave as they should compared to a project that doesn't use Popochiu.

Popochiu should not override this configuration (setup.gd:133, setup.gd:134).

The cause of the problem needs to be identified (it might be something in the camera settings).

Evidence
The gray bar on the left should not be visible if display/window/stretch/aspect is set to expand.

Expected Behavior
The center of the image should always be visible, showing more or less of it as the game window's width changes.

This is related with how Popochiu sets limit to the camera based on the viewport size of the project.
An idea is that could change this by not limiting the camera using its properties but by code. We'll have to research which will be the best approach.

In 4b1317eabbaadaf7190496a66a7ca86e9f0d3734 I made a change so the plugin doesn't override the configuration made by the developer in the project_settings. Still need to check how to handle this since it is related to the limits we are setting to the main camera, which are like they are to make it behave as expected (in terms of following the PC). I'll try to update this for beta 1 or 2.

Moved to 2.1 because solving this will require pretty impacting changes to how camera works.


Does it support Marian machine translation models?

I am trying to prune MarianMT model for example https://huggingface.co/Helsinki-NLP/opus-mt-en-ar. and the library seems not to support it. Is there any way to use it for pruning that model, or what parts of the code should I modify to make it compatible?

Hi this model seems to be a seq2seq model, and you can learn a set of masks to put on both the encoder and decoder side of the model for pruning.


react-apollo is discontinued

Does it support apollo-client?
https://github.com/apollographql/apollo-client

The docs also show how to use it with ApolloLink, which still exists. Did you try that?

The docs also show how to use it with ApolloLink, which still exists. Did you try that?

If I use with Apollo Link it will apply the transformation to every query, and if I want just to a specific query, there a way to do that?


Create a page for the Community Welcomingness metric model

The "Community Welcomingness" metric model is in the GitHub repo but not on the website. The desired state is to have a dedicated page for the Community Welcomingness metric model, like the others listed on the website.
// @geekygirldawn @ElizabethN

This is complete! Not sure what happened here, but thanks for catching this @nwanduka and @geekygirldawn


编译出错

日志如下
{standard input}: Assembler messages:
{standard input}:55079: Warning: end of file not at end of a line; newline inserted
{standard input}:55206: Error: unknown pseudo-op: `.ule'
c++: internal compiler error: Killed (program cc1plus)
Please submit a full bug report,
with preprocessed source if appropriate.
See <file:///usr/share/doc/gcc-7/README.Bugs> for instructions.
make[2]: *** [src/CMakeFiles/OBJ_LIB.dir/brpc/acceptor.cpp.o] Error 4
src/CMakeFiles/OBJ_LIB.dir/build.make:830: recipe for target 'src/CMakeFiles/OBJ_LIB.dir/brpc/acceptor.cpp.o' failed
make[2]: *** Waiting for unfinished jobs....
make[1]: *** [src/CMakeFiles/OBJ_LIB.dir/all] Error 2
make: *** [all] Error 2
CMakeFiles/Makefile2:130: recipe for target 'src/CMakeFiles/OBJ_LIB.dir/all' failed
Makefile:129: recipe for target 'all' failed
Removing intermediate container 5503cda52944
 ---> 856b6b931c94
Step 7/8 : RUN rm -rf _build && mkdir _build && cd _build && cmake .. && make -j8
 ---> Running in 70935faa29ef
-- The C compiler identification is GNU 7.3.0
-- The CXX compiler identification is GNU 7.3.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - not found
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE
-- Found Protobuf: /usr/lib/x86_64-linux-gnu/libprotobuf.so;-lpthread (found version "3.0.0")
CMake Error at CMakeLists.txt:20 (message):
  Fail to find brpc

此外还有提示
debconf: delaying package configuration, since apt-utils is not installed

Fail to find brpc,
确定brpc安装成功了？

@coneycode 我是直接编译docker镜像报的错

Docker配置的内存大小是多少？一般4G内存可以正常编译。

@jiyoushengbd 确实是这样 谢谢了


[Task]: Make resource id search input work

Is there an existing issue for this?

[X] I have searched the existing issues

SubTasks
Refer: https://github.com/appsmithorg/appsmith/issues/15373

Input is working and the value gets stored in the redux store.


Color differences between CPU and GPU decoded frames

From @pjs102793's  https://github.com/pytorch/torchcodec/issues/424#issuecomment-2522514388
to generate sample video: ffmpeg -f lavfi -i testsrc2=size=1920x1080:rate=30 -t 1 testsrc2.mp4
On main b0de666
Repro:

import torch
from torchcodec.decoders import VideoDecoder

cpudec = VideoDecoder("/home/nicolashug/testsrc2.mp4", device="cpu")
gpudec = VideoDecoder("/home/nicolashug/testsrc2.mp4", device="cuda")

cpu_frames = cpudec[:]
gpu_frames = gpudec[:].cpu()

from torchvision.transforms.v2.functional import to_pil_image
from torchvision.utils import make_grid

for i, (cpu, gpu) in enumerate(zip(cpu_frames, gpu_frames)):
    img = to_pil_image(make_grid([cpu, gpu], nrow=2))
    img.save(f"/home/nicolashug/imgs/{i}.jpg")

torch.testing.assert_close(cpu_frames, gpu_frames.cpu())

We're getting pretty high atol differences:
AssertionError: Tensor-likes are not equal!

Mismatched elements: 165091589 / 186624000 (88.5%)
Greatest absolute difference: 48 at index (6, 0, 189, 236)
Greatest relative difference: inf at index (0, 2, 8, 4)

Looking at the decoded frames, they look very similar but the colors are a bit different. So it seems to be a color issue, not an indexing or seeking issue. Example frames (cpu left, gpu right):

Note that it's unlikely to be related or caused by https://github.com/pytorch/torchcodec/pull/413, because I'm observing the exact same differences when reverting that commit.
Thanks again @pjs102793 for the initial report. Do you have any intuition on what might be causing those differences on this specific video? From what I understand, you didn't notice major difference either on other videos, right?

Yes, the videos I have, encoded in H.264 with YUV420p, don’t seem to have any issues. I even tested with the Big Buck Bunny video, and it worked fine. The issue appears to occur only with testsrc2. One possible reason could be that testsrc2 has its colorspace information marked as unknown, which might be causing the problem.
Additionally, when I compared the values between torchaudio on GPU and torchcodec on CPU, the differences were minimal. However, the difference between torchaudio on GPU and torchcodec on GPU was comparable to the difference between torchcodec on CPU and GPU. This suggests that under certain conditions, torchcodec on GPU may produce slightly different colors.


Social elements not rendered horizontally on Outlook mobile web client

Describe the bug
The mobile web version of the Outlook client does not seem to respect the mode="horizontal" attribute on the mj-social element. It stacks the elements vertically instead of displaying them in one row.
To Reproduce
Steps to reproduce the behavior:

Open the MJML application
Paste the following snippet:

<mjml>
  <mj-body>
    <mj-wrapper full-width="full-width">
      <mj-section padding-bottom="0">
        <mj-column padding="0">
          <mj-social font-size="15px" icon-size="30px" mode="horizontal" padding="0" align="center">
            <mj-social-element name="instagram-noshare" href="https://mjml.io/" href="https://mjml.io/">
            </mj-social-element>
            <mj-social-element name="facebook-noshare" href="https://mjml.io/">
            </mj-social-element>
            <mj-social-element name="youtube-noshare" href="https://mjml.io/">
            </mj-social-element>
            <mj-social-element name="twitter-noshare" href="https://mjml.io/">
            </mj-social-element>
          </mj-social>
        </mj-column>
      </mj-section>
    </mj-wrapper>
  </mj-body>
</mjml>

Send the HTML to an outlook address.
Open Chrome and use the developer toolbar to switch to the "mobile device mode"
Login to your Outlook account, open the email and notice that the icons are stacked vertically instead of horizontally

Expected behavior
The social icons are supposed to be rendered horizontally in one single row on both desktop and mobile views.
MJML environment (please complete the following information):

OS: MacOS 10.14.6
MJML Version: v^4.6.2
MJML tool used: MJML App v3.0.2

Email sending environment(for rendering issues):

Platform used to send the email: Mailjet

Affected email clients (for rendering issues):

Email Client: Outlook Mobile on the Web
OS: MacOS
Browser: Chrome in mobile device mode (using dev tools)

Screenshots

I'm not sure which version of outlook you're using, i've just tried it on my account and it renders fine ?

Just use a mobile device or Chrome with mobile device mode enabled and open the email at https://outlook.live.com.

Ok, I can reproduce on iOS Safari. I don't know yet if we'll be able to fix it soon (or find a fix) as mobile Outlook.com isn't really a popular client, people tend to use the App instead of the website.

Well I don't know why this version of outlook have a special media query that force every table to a 100% !important.
Still not sure that we can really override this behavior from outlook, I don't know know if litmus partnership with Microsoft is still a thing https://litmus.com/microsoft-partnership but maybe we can forward this issue to them.

I'll try to get another shot at this maybe with a special media query that force this table to be inherit instead of 100% with https://howtotarget.email/#2019-10-15-outlook-dark-mode

anyone managed to find a solution to this or  a hack to fix it?

I have this same issue. Anyone found a work around yet?