Index additional printer columns for Gatekeeper constraints

The first commit is a slight performance enhancement in the generic informer. It's unrelated to the larger change but I figured I'd include it.
Gatekeeper constraints are CRD based but the CRDs are generated based on
user defined ConstraintTemplate objects. Because of this, indexing the
constraint's spec.enforcementAction and status.totalViolations is a
bit difficult with the current setup. This makes it so that the CRD's
additionalPrinterColumns are automatically indexed.
The scope of this change is limited to the constraints.gatekeeper.sh API
group, but removing that check in the the GenericResourceBuilder
function will enable it for all resources that don't have a custom
transform.
Note that the names from the additionalPrinterColumns in the CRD are
converted to lowerCamelCase style when set as node properties. The
separators are spaces and dashes (e.g. "enforcement action" and
"enforcement-action" become "enforcementAction). Duplicate node
properties are ignored, with a priority on the default generic property
names.
As part of this change, it made sense to also make syncInformers event
driven so that it only gets run when there is a likely relevent change
in a CRD (i.e. added, deleted, or the spec changed). This makes the
indexing of new custom resources nearly instant while also sharing the
CRD informer for keeping an up to date mapping of GVRs to additional
printer columns.
Lastly, main.go was refactored a bit to wait for the two main goroutines
to end before exiting. This allows for some clean up when the interrupt
signal is sent to the process. More refactoring to let every goroutine
clean up should be considered.
Relates:
https://issues.redhat.com/browse/ACM-13278
https://issues.redhat.com/browse/ACM-12572

@jlpadilla this is ready to review, but it's still missing test coverage. Once you agree with how things are implemented/structured, I'll go ahead figure out how I can add the most test coverage possible here.

@jlpadilla I added unit tests, but it's not particularly practical to add a unit test to RunInformers as there is currently no coverage. The test-on-kind-cluster tests should have at least verified that RunInformers gets run successfully but that doesn't count in the test coverage.
What do you recommend?

@jlpadilla the latest push made this also configurable but I'll have to rebase after #302.

@jlpadilla okay, this is ready for review again!

/override ci/prow/sonar-pre-submit
/override SonarCould Code Analysis
Overriding quality gate because it's not feasible to meet the coverage requirement for this change.

/override ci/prow/sonar-pre-submit
/override "SonarCould Code Analysis"
Overriding quality gate because it's not feasible to meet the coverage requirement for this change.

/override ci/prow/sonar-pre-submit
Overriding quality gate because it's not feasible to meet the coverage requirement for this change.

/override "SonarCould Code Analysis"

/override "SonarClould Code Analysis"

/override "SonarCloud Code Analysis"

/override "SonarCloud Code Analysis"


Gentle rotations instead of snapping to orientation

Anchors should be able to handle rotations around an anchor with nice slerp animations. Similar to anchor #1, I just couldn't get the complexities of ConfigurableJoint figured out at the time. This should be addressed again because the snapping is violent and is likely to toss stacked objects around.

I use the joint's rotation mechanisms now but I also cheat a little beforehand in DesiredLocalOrientationFor so I wouldn't call this done quite yet.

Done!


Optimize param validation through new try_get_param

Objective
Fixes #15505
Waiting for #15500, should be merged before this is even finished
Solution
TODO
Testing
TODO

On hold.
It may be possible to use a better solution
https://discord.com/channels/691052431525675048/749335865876021248/1291089012039094303

Closed in favor of #15606


SetUID attack vector space

I would like to contribute to this repo by explaining about attack mentioned above. Please assign me this issue under hacktober fest 2022.
Thank You!

@SANTHOSH17-DOT
Please assign this to me I would like to work on it

@SANTHOSH17-DOT can i add imaged related to this attack

Sure you can add images

code by me will be written in code of conduct? @SANTHOSH17-DOT

No. Create a new directory for this attack and add your work there.

ok @SANTHOSH17-DOT

@SANTHOSH17-DOT  Should i make changes in new branch ? Like named setuid branch

Name of the branch isn't a problem. Make sure that the directory name is "SET UID"

@SANTHOSH17-DOT  ok fine , so changes made by me will be pushed in to the main  branch

like there is no need to create another branch  but the directory name should be SETUID

and is there also need to create markdown file ? @SANTHOSH17-DOT

Yes to describe the research you've made

@SANTHOSH17-DOT  ok but i  am asking that is it necessary to create another branch  or should go with the same branch

You can do anything. But avoid merge conflicts when you raise a PR. I would suggest you to create a new branch

ok @SANTHOSH17-DOT

I would like to contribute to this project. If you think that would be helpful kindly assign this issue to me. Thank You!


rexray - usable to use s3

Summary
not able to use s3 from my raspberry pi
Bug Reports
I have configured using REX-Ray Configuration Generator (http://rexrayconfig.codedellemc.com/) for my raspeberry pi. now when I try to start the service am getting error:
error: service startup failed: libstorage: invalid format
It should throw the right error message - why an invalid format ? when I am getting the config.yml generated from the configuration generator tool.
Version
pi@raspberrypi2:~/Downloads $ rexray version
REX-Ray
Binary: /usr/bin/rexray
Flavor: client+agent+controller
SemVer: 0.11.0+8
OsArch: Linux-ARMv7
Commit: c9d10b92e030e872a45d92aa64c33cf30621a0de
Formed: Wed, 25 Oct 2017 23:12:42 UTC
Expected Behavior
when I am using s3fs, am able to access the S3 bucket, but through rexray am not able to access the S3 bucket, which I should have.
Actual Behavior
I should be able to access my S3 bucket.
Steps To Reproduce
I used the step to configure provided at
https://rexray.readthedocs.io/en/stable/
in configure section.
Copy and paste the contents from the REX-Ray Configuration Generator to a new file on the host at /etc/rexray/config.yml.
 It looks like the spaces have been squashed to indicate
child items?
On Fri, Oct 27, 2017 at 5:15 PM udayansr<EMAIL_ADDRESS>wrote:

Summary
not able to use s3 from my raspberry pi
Bug Reports
I have configured using REX-Ray Configuration Generator (
http://rexrayconfig.codedellemc.com/) for my raspeberry pi. now when I
try to start the service am getting error:
error: service startup failed: libstorage: invalid format
It should throw the right error message - why an invalid format ? when I
am getting the config.yml generated from the configuration generator tool.
Version pi@raspberrypi2:~/Downloads $ rexray version
REX-Ray
Binary: /usr/bin/rexray
Flavor: client+agent+controller
SemVer: 0.11.0+8
OsArch: Linux-ARMv7
Commit: c9d10b9
https://github.com/thecodeteam/rexray/commit/c9d10b92e030e872a45d92aa64c33cf30621a0de
Formed: Wed, 25 Oct 2017 23:12:42 UTC
Expected Behavior
when I am using s3fs, am able to access the S3 bucket, but through rexray
am not able to access the S3 bucket, which I should have.
Actual Behavior
I should be able to access my S3 bucket.
Steps To Reproduce
I used the step to configure provided at
https://rexray.readthedocs.io/en/stable/
in configure section.
Copy and paste the contents from the REX-Ray Configuration Generator to a
new file on the host at /etc/rexray/config.yml.
Configuration Files
pi@raspberrypi2:~/Downloads $ cat /etc/rexray/config.yml
rexray:
logLevel: info
libstorage:
service: s3fs
integration:
volume:
operations:
create:
default:
size: 1
s3fs:
accessKey: AKIAIRQK25YYR27BPUAA
secretKey: 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
region: ap-south-1
Client Log
clientlog.txt
https://github.com/thecodeteam/rexray/files/1422502/clientlog.txt
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/thecodeteam/rexray/issues/1097, or mute the thread
https://github.com/notifications/unsubscribe-auth/ABVMMUT3J2JalFtCT1I22L57YrG5OaIQks5swfN_gaJpZM4QJNTy
.

@clintkitson , its the one that i downloaded from EX-Ray Configuration Generator (http://rexrayconfig.codedellemc.com/)
Didnt edit it, so no changes done by me manually.
Here is the one that I have used:
config.yml.txt
Thanks,
US

Interesting.. it looks like for some reason it is defaulting to look under
a different directory for the config.
Try specifying it manually with -c /etc/rexray/ config.yml
On Fri, Oct 27, 2017 at 5:37 PM udayansr<EMAIL_ADDRESS>wrote:

@clintkitson https://github.com/clintkitson , its the one that i
downloaded from EX-Ray Configuration Generator (
http://rexrayconfig.codedellemc.com/)
Didnt edit it, so no changes done by me manually.
Here is the one that I have used:
config.yml.txt
https://github.com/thecodeteam/rexray/files/1422573/config.yml.txt
Thanks,
US
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/thecodeteam/rexray/issues/1097#issuecomment-340006609,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABVMMRO-kD0b2UO_ENlrUqbTVrygZFFEks5swfivgaJpZM4QJNTy
.

Hi @clintkitson ,
I tried it, but still doesnt work. Here is the log message with debug enabled.
client_log_config_specific.txt
Thanks,
US

Interesting..
@akutz can you take a peak?
For some reason the base path for the PI is defaulting to /home/pi/.rexray
which is causing the docker unix socket to try and be created in a
directory that doesn't exist and not relevant to docker.
On Fri, Oct 27, 2017 at 5:58 PM udayansr<EMAIL_ADDRESS>wrote:

Hi @clintkitson https://github.com/clintkitson ,
I tried it, but still doesnt work. Here is the log message with debug
enabled.
client_log_config_specific.txt
https://github.com/thecodeteam/rexray/files/1422642/client_log_config_specific.txt
Thanks,
US
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/thecodeteam/rexray/issues/1097#issuecomment-340012421,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABVMMWt5gZ7cUH_jhOncmaBu_kZmtJWHks5swf2rgaJpZM4QJNTy
.

Hi @akutz (cc @clintkitson ),
I again tried, now here is an interesting part:
From docker volume related commands, I can do:
docker_volume_cmds.txt
While I try the command set using rexray, it doesnt work:
rexray_volume_cmd.txt
Need your help here, to resolve. Let me know in case you need any other file.
Thanks,
US

Specify -c /etc/config.yml after rexray commands.
On Sat, Oct 28, 2017 at 3:32 AM udayansr<EMAIL_ADDRESS>wrote:

Hi @akutz https://github.com/akutz (cc @clintkitson
https://github.com/clintkitson ),
I again tried, now here is an interesting part:
From docker volume related commands, I can do:
docker_volume_cmds.txt
https://github.com/thecodeteam/rexray/files/1424142/docker_volume_cmds.txt
While I try the command set using rexray, it doesnt work:
rexray_volume_cmd.txt
https://github.com/thecodeteam/rexray/files/1424145/rexray_volume_cmd.txt
Need your help here, to resolve. Let me know in case you need any other
file.
Thanks,
US
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/thecodeteam/rexray/issues/1097#issuecomment-340173824,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABVMMaSU43zOg2w7U_CmGIuXyqIyVI3Iks5swwK6gaJpZM4QJNTy
.

Hi @clintkitson , it works.
Still the question on how to fix the directory problem through rexray env variable or any other way ? Else, i need to use -c /etc/rexray/config.yml as option always ?
Thanks,
US

It's something specific to Pi, we will take a peak and update this when we
identify why.
On Sat, Oct 28, 2017 at 10:33 AM udayansr<EMAIL_ADDRESS>wrote:

Hi @clintkitson https://github.com/clintkitson , it works.
Still the question on how to fix the directory problem through rexray env
variable or any other way ? Else, i need to use -c /etc/rexray/config.yml
as option always ?
Thanks,
US
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/thecodeteam/rexray/issues/1097#issuecomment-340207289,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABVMMTtvaiqnxF-eLVgluli-3lmuuIMiks5sw2VegaJpZM4QJNTy
.

Hi!
I have the same problem, but using the azureud driver.
With the -c option it works.

Thanks for the feedback @dopic, will look at this.

Did you solve this issue now?
I have the same issue now, even I specify the config.yml path with -c /etc/rexray/config.yml , the problem is also happened,  I don't know how to solve this...
waiting for your reply~


watir-webdriver + firefox is very slow to loop table rows

I'm on OS X 10.10.2. I've tried all the possible combinations of these versions:
watir-webdriver: 0.6.11, 0.7.0
Firefox: 33.03, 34.0.5, 36.0.4
but regardless of the versions, the following loop is painfully slow:
browser.table(id: 'table').tbody.trs.each { |tr| puts tr.id }
and
browser.table(id: 'table').tbody.trs.to_a.each { |tr| puts tr.id }
20 rows takes about 20 seconds to loop like this.
Is it possible to make this loop faster? What is causing it to be so slow? Thanks!

Yes, unfortunately this is a Selenium bug in version 2.45- https://code.google.com/p/selenium/issues/detail?id=8551
Firefox before version 36 and using selenium-webdriver 2.45.0.dev3 should work

@titusfortner Ah, thanks. Are you saying watir-webdriver is currently unusable or how does the selenium-webdriver fix this (I'm unfamiliar with selenium-webdriver)? Are we supposed to simply include the selenium-webdriver in the Gemfile? Thanks again!

To answer my own question: add the following to your Gemfile
gem 'selenium-webdriver', '2.45.0.dev3'

After downgrading to FF34 and selenium-webdriver 2.45.0.dev3, the loop still takes 10 seconds.
Is there a way to force a "prefetch" so that the information would be cached and not piped into the browser everytime?
Right now I'm thinking doing this:
doc = Nokogiri::HTML(browser.table(id: 'table').html)
and using Nokogiri to scrape the information in much faster manner. Am I think along the right lines here? Thanks!

How fast does it take you in Chrome?
Do you notice a difference between 0.6.11 and 0.7? (we changed some things that could affect speed)

Also, @jkotests has this project already, based on @zeljkofilipin's idea: https://jkotests.wordpress.com/2013/08/21/watir-nokogiri-gem-published/
I kind of think it would be interesting to bake this into Watir itself in a way that doesn't require a duplicate project. I see Watir more as a testing framework than a WebDriver DSL. As such, Watir shouldn't *need to use WebDriver for things that would be much faster/better with an HTML library.
I'm more focused right now on improving Watir's reliability as a test framework than digging into improving performance, but it is an interesting idea.

There is a big difference between using nokogiri and watir: with nokogiri you can retrieve any data you need (+ there are tons of other stuff for that like mechanize), with watir - you get only what end user gets. For instance, if element is invisible, #text will return empty string which is the same as user sees, while nokogiri would still returns non-empty string.
Having a separate watir-nokogiri or watir-mechanize seems like a great idea: it's a sort of new celeriy.

Thank you guys for this enlightening discussion.
@titusfortner Unfortunately 0.7.0 requires selenium-webdriver (>= 2.45), so I couldn't test 0.7.0 with 2.45.0.dev3. 0.6.11 with 2.45.0.dev3 definitely feels much faster than 0.7.0 with 2.45.0 (but that could very well be to the mentioned Selenium bug).
I can totally understand the benefits of watir being always in sync with the real browser state (especially important in end-to-end testing among other things). I would argue, however, that from what I've personally experienced and seen other people go through, providing a watir-nokogiri like experience would make sense right in this gem. The performance issues seem to be relevant a lot of times. The tradeoffs would be needed to be documented very clearly, sure.
So +1 for bringing watir-nokogiri implementation to watir-webdriver. Thanks again!

I just realized that tbody and trs in a chain like browser.table(id: 'table').tbody.trs probably traverses the whole DOM sub-tree instead of just immediate children. If the sub-trees are deep and large, then it's not that surprising that these kind of queries are slow.
I'm curious to experiment what effect would limiting the search to immediate children have. Is there a way to restrict the search to just immediate children (would you need to revert to using css/xpath selectors) and do you think this could have meaningful effect?

What do you mean by traversing the whole sub-tree vs immediate children?
The chain is the same as locating all elements with CSS table#table tbody tr.

I think this author does a lot better job explaining what I'm trying to say http://www.natontesting.com/2009/08/19/how-to-get-only-direct-child-objects-in-watir/
My example was pretty poor (well, unless the table uses nested tables).

I see what you mean. You can still use XPath/CSS to retrieve first childs:
browser.trs(xpath: "//table[@id='table']/tbody/tr")

I would say that in most cases you should just limit the locating elements (e.g. by adding class selector), I'd argue that retrieving the first child is more expected behavior - current implementation is much more flexible (since allows to use both approaches).

Cool, that works if we begin from the document root and/or have an absolute reference point. But often times we are searching relative to a Watir Element (because we need the parent element as well as the children). So, how would you conveniently limit the search scope to the immediate children in this case?
table = browser.table(id: 'table')
table.trs.each do |tr| # problem, not immediate
  tr.tds.each do |td| # problem, not immediate
    # ...
  end
end

I would just limit the lookup by adding classes to trs/tds.
On the other hand, I see the problem you are talking about even though I have personally never had any problems with it. I see always locating the immediate child has a lot of problems (for example, you have to follow the whole chain) but it would be great other people thoughts on this.

I've been working a lot on deeply nested elements recently. 0.6.11 actually does not support certain forms of nesting at all. This was fixed for 0.7.0, but at the expense of some performance. My current pull request fixes that performance significantly for iterating over deeply nested elements, but it shouldn't affect performance issues you might be having with 0.6.11.
A quick example of how Watir will process this in the next version:
trs = browser.table.trs.to_a
trs.each do |tr|
  tds = tr.tds.to_a
  tds.each do |td|
    puts td.text
  end
end

browser.table.trs.to_a locates all of the trs on the page under the table with wire calls like this:

Verify window isn't closed --> getWindowHandles & getCurrentWindowHandle
Ensure driver is in top level browsing context --> switchToFrame
Look up table --> findElement
Verify Selenium Element is a table --> getElementTagName
Look up all trs --> findChildElements

For each tr, tr.td.to_a does:

Verify parent isn't stale --> isElementEnabled
Look up child elements --> findChildElements

For each td.text:

Verify td exists --> isElementEnabled
Get text --> getElementText

Total wire calls:
6+2trs.size+2(total_number_tds)
So, there aren't any superfluous calls to the driver here, as each call gives important information to the overall test.
I have not worked with xpath selectors much. I think you should still be able to nest them the same way:
browser.table(id: 'table').trs(xpath: "/tbody/tr").each { |tr| tr.tds(xpath: "/td"). each { |td| puts td.text} }
It's implemented with: @wd.find_elements(:xpath, XPATH_STRING)[idx]
and @wd would be the Selenium::WebDriver::Element of the parent, so it should only look at specific children of that element each call.
Are you noticing the same performance issue in Chrome? It could be a limitation for the Firefox driver itself, which will hopefully be reduced when Mozilla finally releases Marionette.

Switching to Chrome didn't seem to have any significant impact.
Using trs(xpath: './*') and tds(xpath: './*')was slower than justtrsandtds`.
tbody.trs(xpath: '/tr') returns no matches (the selector must be wrong).
I got it working by using tbody.trs(xpath: './tr') and `tr.tds(xpath: './td')', but this also didn't have any significant impact it seems.
Hmm, would be pretty neat if we could have a standard test website to benchmark against.


SSH & emacs cause phantom characters when trying to scroll down

Your Windows build number: (Type ver at a Windows Command Prompt)
Version 10.0.16299.309

What you're doing and what's happening: (Copy & paste specific commands and their output, or include screen shots)
Using ssh, connect to a system and open a file using emacs, as you open the file press the down key to scroll.

What's wrong / what should be happening instead:
The file should open a scroll down, instead it adds a B to the top of the text file while it loads.

No entriley sure this is a console issue but the same issue dose not apply when using wsltty. Only seems to be an issue with emacs, I guess VI isn't in insert mode by default therefore it might not affect it.

What's TERM set to on the remote machine? A lot of issues like these are because the remote machine is set to something other than xterm-256color

Spot on
echo $TERM
xterm-256color
I guess the terminal has a general bug with xterm-256color? Or is there an altertive we should be setting.
xterm-256color must be the default for RHEL, CentOS etc.

Im using vi with RHEL/SLES/Ubuntu, and got same problem (https://github.com/Microsoft/console/issues/188#issuecomment-391362285).
When I set => TERM="" => cursor key works, but page up/down not.
With TERM=xterm-color => page up/down works, cursor keys not
I could remap it in vimrc, but Im too lazy :-). With these bugs this m$ console is useless for me. Use now kitty (putty) again :-) ....

Hmm I see that this issue is a year old, but I still have the same issue with emacs on CentOS 7.8 in Windows Terminal.
When googling I found a thread for another console/terminal program ConEmu which had a similar issue with emacs: https://github.com/Maximus5/ConEmu/issues/1691
The top comment there seems to have some more detailed information about what the reason could be. Does that help in getting the issue fixed for Windows Terminal?

Hey this issue is... pretty dang old at this point. Anyone still seeing this? I reckon this is probably due to an input sequence getting chopped in half and not getting written to the console in a single write. Can we get a fresh repro here with a specific ssh version (and distribution)/?

i haven't had it since i switch to:
https://github.com/microsoft/terminal/issues/173#issuecomment-388091934
Thanks,

That's kinda weird tbh, cause typically the Windows Console & Terminal should work with xterm-256color. I suppose your issue is sorted out, and we can just close this out for now. If something else like this comes up, we can track it in a fresh issue rather than this fairly stale one.
Thanks!


Work in search results

Hello!
Can this extension works on page with project search results?

Thanks.

It looks like it works for me. Is it not working for you?

No. I works fine in files, but not in Project search or Settings.
Arch Linux.
> atom -v
Atom    : 1.32.2
Electron: 2.0.12
Chrome  : 61.0.3163.100
Node    : 8.9.3

Looks like Atom allows this by default on project find and settings on Windows.
I'll look into enabling this package on those elements.

Maybe. I like this feature in Chrome on Windows, but Chrome on Linux doesn't have it. Maybe it's from Electron. I'll be glad to see full support in Atom on Linux, thank you.

In files tree doesn't work too…


update version on changelog

Reason for change

Testing

Checklist

[ ] CHANGELOG: The Unreleased section of CHANGELOG was updated
[ ] Prop changes: docs were updated
[ ] Prop changes: E2E tests were updated, testing from the highest level possible
[ ] Platform testing: If this change should be tested against a platform, has it been?

Once this RC publishes I’ll test another runthrough and ✅

@davidleger95 We should have merged this before releasing, because the CHANGELOG for 0.5.14 isn’t up-to-date: https://unpkg.com/browse/<EMAIL_ADDRESS>
1 | # Changelog
-- | --
2 |
3 | All notable changes to this project will be documented in this file.
4 |
5 | The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project
6 | adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
7 |
8 | ## [Unreleased]
9 |
10 | ## Added
11 |
12 | - Added `disabled` attribute to `manifold-resource-rename` (#553)
13 |
14 | ### Fixed
15 |
16 | - Fixed local testing addresses.
17 | - Fixed console warning about `loading="lazy"` (#552)
18 | - Prevent actions on `<manifold-credentials>` while awaiting a resourceLabel (#554)
19 |
20 | ### Changed
21 |
22 | - Changed number of skeleton products that load (#551)
23 |
24 | ## [v0.5.13]

We should have merged this before releasing

@dangodev my bad, for some reason I thought you had merged it


InsufficientBalance error trying to swap USDC

Hi, I was looking to swap USDC into DAI
import sys

from uniswap import Uniswap
from web3 import Web3
from web3.exceptions import InvalidAddress
from web3.gas_strategies.time_based import fast_gas_price_strategy
from web3.middleware import geth_poa_middleware

# Define web3 provider
rpc_url = 'https://endpoints.omniatech.io/v1/arbitrum/one/public'
web3 = Web3(Web3.HTTPProvider(rpc_url))
web3.middleware_onion.inject(geth_poa_middleware, layer=0)
web3.eth.set_gas_price_strategy(fast_gas_price_strategy)

dai_address = '0xDA10009cBd5D07dd0CeCc66161FC93D7c9000da1'
usdc_address = '0xFF970A61A04b1cA14834A43f5dE4533eBDDB5CC8'

base_address = usdc_address
quote_address = dai_address

try:
    uniswap_client = Uniswap(
        address='<MY_ADDRESS>',
        private_key='<MY_PRIVATE_KEY>',
        web3=web3,
        version=3)
except InvalidAddress:
    print('Invalid Uniswap contract address')
    sys.exit(-1)

try:
    txn = uniswap_client.make_trade(
        base_address,
        quote_address,
        20 * 10 ** 18, # swap 20 USDC into DAI
    )
    print(f'Transaction hash: {txn.hex()}')
except ValueError as e:
    print(f'Error: {e}')

Error:
uniswap.exceptions.InsufficientBalance: Insufficient balance. Had 12....., needed<PHONE_NUMBER>0000000000

so If I want to convert 1 ETH that would be 1 * 10 ** 18. What about other tokens? reading the docs seemed to me 1 unit of any token is always passed as 1 * 10 ** 18, apparently, it is not.

The number of decimals depends on the token. ETH has 18, but USDC uses 6.
The number is specified in the ERC-20 contract of the token, so can be
looked up on Etherscan etc.
On Sun, May 14, 2023, 20:12 MelarDev @.***> wrote:

Hi, I was looking to swap USDC into DAI
import sys
from uniswap import Uniswapfrom web3 import Web3from web3.exceptions import InvalidAddressfrom web3.gas_strategies.time_based import fast_gas_price_strategyfrom web3.middleware import geth_poa_middleware
Define web3 providerrpc_url = 'https://endpoints.omniatech.io/v1/arbitrum/one/public'web3 = Web3(Web3.HTTPProvider(rpc_url))web3.middleware_onion.inject(geth_poa_middleware, layer=0)web3.eth.set_gas_price_strategy(fast_gas_price_strategy)
dai_address = '0xDA10009cBd5D07dd0CeCc66161FC93D7c9000da1'usdc_address = '0xFF970A61A04b1cA14834A43f5dE4533eBDDB5CC8'
base_address = usdc_addressquote_address = dai_address
try:
uniswap_client = Uniswap(
address='<MY_ADDRESS>',
private_key='<MY_PRIVATE_KEY>',
web3=web3,
version=3)except InvalidAddress:
print('Invalid Uniswap contract address')
sys.exit(-1)
try:
txn = uniswap_client.make_trade(
base_address,
quote_address,
20 * 10 ** 18, # swap 20 USDC into DAI
)
print(f'Transaction hash: {txn.hex()}')except ValueError as e:
print(f'Error: {e}')
Error:
uniswap.exceptions.InsufficientBalance: Insufficient balance. Had 12....., needed<PHONE_NUMBER>0000000000
so If I want to convert 1 ETH that would be 1 * 10 ** 18. What about other
tokens? reading the docs seemed to me 1 unit of any token is always passed
as 1 * 10 ** 18, apparently, it is not.
—
Reply to this email directly, view it on GitHub
https://github.com/uniswap-python/uniswap-python/issues/333, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AAKXDOXC7ZZO7SB4GKJDMH3XGEN73ANCNFSM6AAAAAAYBJWIDM
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>


Reintroduce edit/cancel buttons for questions

Fixes #1502

Updated questions 'index' partial to always display questions as 'show' on section load
Added a 'Edit' button to question 'show' partial
Added a 'Cancel' button to question 'form' partial
Updated JS to load 'edit', 'new' and 'show' partials via AJAX when the user clicks the above buttons. For some reason Rails' remote: true does not work here (maybe a name/selector collision with other remote forms/links elsewhere on the page?)
Added 'Loading...' and failure messages for instances where AJAX is loading the form or is unable to load the form

@jollopre I'll need you to take a look at this one. I'm not seeing what the issue is. Scenario (you can test on roadmap-stg):

Open a template phase for editing
Expand a section (all questions should now display as 'show' partials with a new 'Edit' button)
Click 'Edit' on one of the questions ('show' partial should be replaced with 'edit' partial)
Scroll to the bottom of the form and click 'Cancel' ('show' partial should get reloaded)
Click 'Edit' again on the same question ('edit' partial reloads but Tinymce editors are not loaded)

The Tinymce editors get wired up properly the first time the questions 'form' partial is loaded when you click 'Edit' (same is true for 'Add question') but not on subsequent loads. All other JS is working properly.
I tried calling the Tinymce.destroy... methods when the user clicks 'Cancel' on the form before replacing the contents of the container but it didn't help.


OData for Graph API

Do some research for OData:

https://github.com/OData/odata.net
http://odata.github.io
https://visualstudiogallery.msdn.microsoft.com/9b786c0e-79d1-4a50-89a5-125e57475937
http://blogs.msdn.com/b/odatateam/archive/2014/03/11/how-to-use-odata-client-code-generator-to-generate-client-side-proxy-class.aspx

Graph API schema is basically CSDL – OData format. Need to find a way to utilise OData for Graph API

Done


TF1 is not working anymore

Checklist

[x] I'm reporting a broken site support
[x] I've verified that I'm running youtube-dl version 2021.12.17
[x] I've checked that all provided URLs are alive and playable in a browser
[x] I've checked that all URLs and arguments with special characters are properly quoted or escaped
[x] I've searched the bugtracker for similar issues including closed ones

Verbose log

$ youtube-dl "https://www.tf1.fr/tf1/koh-lanta/videos/koh-lanta-le-totem-maudit-du-31-mai-2022-episode-14-31800896.html" --verbose
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['https://www.tf1.fr/tf1/koh-lanta/videos/koh-lanta-le-totem-maudit-du-31-mai-2022-episode-14-31800896.html', '--verbose']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, pref UTF-8
[debug] youtube-dl version 2021.12.17
[debug] Python version 3.10.1 (CPython) - macOS-10.15.7-x86_64-i386-64bit
[debug] exe versions: ffmpeg 4.4.1, ffprobe 4.4.1, rtmpdump 2.4
[debug] Proxy map: {}
[TF1] koh-lanta-le-totem-maudit-du-31-mai-2022-episode-14-31800896: Downloading JSON metadata
[wat.tv] 13885430: Downloading JSON metadata
[wat.tv] 13885430: Downloading MPD manifest
WARNING: Failed to download MPD manifest: HTTP Error 404: Not Found
[wat.tv] 13885430: Downloading JSON metadata
WARNING: Unable to download JSON metadata: HTTP Error 403: Unauthorized access.
ERROR: No video formats found; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Traceback (most recent call last):
  File "/usr/local/Cellar/youtube-dl/2021.12.17/libexec/lib/python3.10/site-packages/youtube_dl/YoutubeDL.py", line 815, in wrapper
    return func(self, *args, **kwargs)
  File "/usr/local/Cellar/youtube-dl/2021.12.17/libexec/lib/python3.10/site-packages/youtube_dl/YoutubeDL.py", line 836, in __extract_info
    ie_result = ie.extract(url)
  File "/usr/local/Cellar/youtube-dl/2021.12.17/libexec/lib/python3.10/site-packages/youtube_dl/extractor/common.py", line 534, in extract
    ie_result = self._real_extract(url)
  File "/usr/local/Cellar/youtube-dl/2021.12.17/libexec/lib/python3.10/site-packages/youtube_dl/extractor/wat.py", line 96, in _real_extract
    self._sort_formats(formats)
  File "/usr/local/Cellar/youtube-dl/2021.12.17/libexec/lib/python3.10/site-packages/youtube_dl/extractor/common.py", line 1374, in _sort_formats
    raise ExtractorError('No video formats found')
youtube_dl.utils.ExtractorError: No video formats found; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.

Description

This used to work but since a few days it seems that something change on the service and I have the error displayed in the log.

This is not duplicate of https://github.com/ytdl-org/youtube-dl/issues/30959 which is for attempting to download the livestream.
TF1 have recently closed their replays and require the user to log in in order to watch them hence the 403 from above.

The report may not be the same, but the underlying issue was.

Resolved in https://github.com/dirkf/youtube-dl/commit/d2d90874d4008238c79e09b3967309809cec037a.


Hotfix/fasten brick cloning

Hi rubocop

Hi @noelle Could you please triple check this changes? Thank you so much!

:+1:  very cool!


luaposix seems to be crashing my app

https://github.com/karai17/lapis-chan/issues/57
https://github.com/luaposix/luaposix/blob/1332cd1452f48b2dfc3b2d5c8a724c16d720a37d/lib/posix/deprecated.lua#L300

Took a bit more fiddling to get it right than I had expected, but got there in the end :)
Thanks for catching the error!

It looks like there is another related issue~
https://github.com/karai17/lapis-chan/issues/57#issuecomment-419765383


timezone conflict

There is a problem with timezone conversion. For example https://notebooks.gesis.org/binderlaunches/#?from=2021-03-01T00%3A00&to=2021-03-31T23%3A59&desc=true&repo=%3A must return launches happened in march, but it shows launches until "4/1/2021, 1:59:00 AM"

Note that datetime format in url changed: https://notebooks.gesis.org/binderlaunches/#?from=2021-02-28T23%3A00%3A00.000Z&to=2021-03-31T21%3A59%3A00.000Z&desc=true&repo=%3A


Support for "stat" panel

So is the support for those panels implemented or still a todo?

A "singlestat" panel has been implemented, but I think it's actually deprecated and silently converted to a "stat" panel by Grafana.
So in terms of feature, grabana supports "stat" panels but there is some work to be done to make sure we're fully up-to-date in terms of what the library generates.

Fixed by #170


FreemarkerTemplateProcessor: Fix-up an error message

Merge regardless of the unrelated failing BowerFunTest on appveyor.

Merge regardless of the unrelated failing BowerFunTest on appveyor.


Add OpenAI batch feature

Might be a new module for openai, optimized batch processing only

We want the result ASAP. So, it doesn't have to implement this feature at AutoRAG.
"Batch feature needs to wait max 24 hour"


PyFloat overflow handling

Codecov Report

Merging #893 into master will decrease coverage by 0.04%.
The diff coverage is 62.68%.

@@            Coverage Diff             @@
##           master     #893      +/-   ##
==========================================
- Coverage   63.93%   63.88%   -0.05%
==========================================
  Files          88       88
  Lines       14880    14852      -28
  Branches     3311     3307       -4
==========================================
- Hits         9513     9488      -25
+ Misses       3194     3180      -14
- Partials     2173     2184      +11

Impacted Files
Coverage Δ

vm/src/obj/objfloat.rs
72.19% <62.68%> (-2.53%)
:arrow_down:

vm/src/obj/objint.rs
86.1% <0%> (+0.67%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e3e2e7f...a0056b0. Read the comment docs.


SCP*_HP settings do not work

I doubled the SCP's health to test this, and then spawned myself in as each of the SCPs as someone read my health to me each time. For every single SCP, I had the default health.
SCP049_HP: 2400
SCP049-2_HP: 800
SCP096_HP: 4000
SCP106_HP: 1400
SCP173_HP: 4000

What version of ServerMod are you using?

Definitely ServerMod 2.1.1. I know because I have my Reserved Slots.txt file set up and my Patrons are able to join the dedicated slots.

ServerMod 2.1.2 is what you want if you're running 6.0.0.b, I'd be surprised if you could run that version on 6.0.0.b

Where's the download for ServerMod 2.1.2? It shows 2.1.1 as the latest release.

Try 2.0.0 and tell me if it still doesn't work

@Dankrushen I'm running 2.2.0 and it still doesn't work. I just tested as SCP-049 and SCP-173 and they have the default health.

Newest SMod should fix this, it was a problem with config beeing not lowercase. New smod doesnt care for lowercase/highercase config

Should be fixed with 2.2.0 re-upload

Is that fix also in the re-uploaded 2.2.1? Because that's the version I'm running and it still seems to be broken. Now that I know the issue, I just replaced all the SCP configurations with their lowercase counterparts.
sed -i 's/SCP049_HP/scp049_hp/g' */config.txt
sed -i 's/SCP049-2_HP/scp049-2_hp/g' */config.txt
sed -i 's/SCP096_HP/scp096_hp/g' */config.txt
sed -i 's/SCP106_HP/scp106_hp/g' */config.txt
sed -i 's/SCP173_HP/scp173_hp/g' */config.txt


solo: Block on starting ZooKeeper

Ensure that ZooKeeper is started before starting the agent or master. This
will help mitigate issues where the agent or master starts up too soon and
can't connect to ZK.

@gimaker

:+1:


Error in Network Readme

Hello,
I noticed an error in the instructions for setting up AirportItlwm. Step 6 is currently:

Open /EFI/OC/config.plist and find the following code:

<dict>
	<key>Arch</key>
	<string>x86_64</string>
	<key>BundlePath</key>
	<string>AirportItlwm.kext</string>
	<key>Comment</key>
	<string>Intel Wi-Fi driver</string>
	<key>Enabled</key>
	<false/>
	...

and change to:
<dict>
	<key>Arch</key>
	<string>x86_64</string>
	<key>BundlePath</key>
	<string>AirportItlwm.kext</string>
	<key>Comment</key>
	<string>Intel Wi-Fi driver</string>
	<key>Enabled</key>
	<true/>
	...

However, it should be:

Open /EFI/OC/config.plist and find the following code:

<dict>
	<key>Arch</key>
	<string>x86_64</string>
	<key>BundlePath</key>
	<string>itlwm.kext</string>
	<key>Comment</key>
	<string>Intel Wi-Fi driver</string>`
	<key>Enabled</key>
	<false/>
	<key>ExecutablePath</key>
	<string>Contents/MacOS/itlwm</string>
	...

and change to:
<dict>
	<key>Arch</key>
	<string>x86_64</string>
	<key>BundlePath</key>
	<string>AirportItlwm.kext</string>
	<key>Comment</key>
	<string>Intel Wi-Fi driver</string>
	<key>Enabled</key>
	<true/>
	<key>ExecutablePath</key>
	<string>Contents/MacOS/AirportItlwm</string>
	...

As it currently reads, the instructions do not work and result in a softlock at the Apple logo. This fixes that.

I recently updated to the big sur beta and I have been experiencing some weird boot issues. I dont know if it is related to AirportItlwm but i updated to AirportItlwm with your recommended settings but it didnt help. The issue seem to be related to itlwm when looking at the verbose output on boot.
My latest boot logs
opencore-2020-09-19-012409.txt
opencore-2020-09-19-013153.txt

@brian-xu Hi, in actual config.plist we have both itlwm and AirportItlwm kexts with Enabled value set to false:
so if you want to use itlwm.kext/AirportItlwm.kext you need to change only its value to true leaving the other unchanged (i.e. set to false)

Just updated Network/Readme and OC/config.plist: we can force injection of IO80211Family.kext for AirportItlwm.kext

Oh, I must have not noticed the AirportItlwm option in the plist. Marking as fixed then.


Can not wipe individual items

Describe the bug
can not wipe individual items
To Reproduce
on load:
wipe crafting recipes for any diamond armor
wipe crafting recipes for any diamond tools
wipe crafting recipes for diamond sword
wipe crafting recipes for enchantment table
wipe crafting recipes for brewing stand
wipe crafting recipes for magma cream
Information:

Skript version: 4.1.4
Server software: Spigot
Server version: 1.12.2
java.lang.NoSuchMethodException: net.minecraft.server.v1_12_R1.CraftingManager.getInstance(). i assume this has to do with the crafting recipe manager.

Additional context
i can create recipes and "wipe server crafting recipes" works but wiping individual recipes like this doesnt work:
wipe crafting recipes for any diamond armor
wipe crafting recipes for any diamond tools
wipe crafting recipes for diamond sword
wipe crafting recipes for enchantment table
wipe crafting recipes for brewing stand
wipe crafting recipes for magma cream
this also gives no error

Crafting recipes were removed, please use any other addon that supports recipes like SkBee


many-to-many relationships without a joining entity

I know this probably gets asked a lot, but is adding support for many-to-many relationships without a joining entity on the roadmap?  If so where?
Jason

How do u imagine many to many withour midle entity ?
By concept many many must use 3 entity !

It does currently in EF 6.x and previous versions of EF as well as in Resharper. I know linking tables are necessary in relational databases for many-to-many relationships, but they dont make a lot of logical in entities.

The ef core is some kind new orm library! I think they not gona do some incorect errorrs in conceptual architecture!

Could U create ERD diagram with table relationship ?
The Third table is need not for modern-fashionable style. 3 table is justified for perfomance and oprtimization purposes!
The current Firebird version database actually still read all space from Disk.
3-rd table this some kind compromise of perfomance vs comfortable usage!

@zanate4019 to your question, the work item for this is https://github.com/aspnet/EntityFrameworkCore/issues/10508.


List Full Course Title

User Story
As a user I can see the full course title in the table so that I can be sure about which course I am looking at.
Acceptance Criteria

[ ] Every course should have a full course title

Implementation Todos
Front end:

[ ] Modify the proper component to display the full course title

Back end:

No changes needed

Testing:

[ ] Front end tests pass and there is adequate coverage
[ ] Back end tests pass and there is adequate coverage

I'm sorry, I believed there was a full course title listed in the JSON--i was wrong. For now you can work on the other 3 issues on your Kanban board.


Error: UNKNOWN_CODE_PLEASE_REPORT: Connection was killed

https://sentry.iobroker.net/share/issue/bfbf00157f54442ea5e04695ae4cf661/
Error: UNKNOWN_CODE_PLEASE_REPORT: Connection was killed
  File "/opt/iobroker/node_modules/mysql/lib/protocol/sequences/Sequence.js", line 47, col 14, in Quit.Sequence._packetToError
    var err  = new Error(code + ': ' + packet.message);
  File "/opt/iobroker/node_modules/mysql/lib/protocol/sequences/Sequence.js", line 96, col 17, in Quit.Sequence.ErrorPacket
    this.end(this._packetToError(packet));
  File "/opt/iobroker/node_modules/mysql/lib/protocol/Protocol.js", line 291, col 23, in Protocol._parsePacket
    sequence[packetName](packet);
  File "/opt/iobroker/node_modules/mysql/lib/protocol/Parser.js", line 433, col 10, in Parser._parsePacket
    this._onPacket(packetHeader);
  File "/opt/iobroker/node_modules/mysql/lib/protocol/Parser.js", line 43, col 10, in Parser.write
    this._parsePacket(packetHeader);
  File "/opt/iobroker/node_modules/mysql/lib/protocol/Protocol.js", line 38, col 16, in Protocol.write
    this._parser.write(buffer);
  File "/opt/iobroker/node_modules/mysql/lib/Connection.js", line 88, col 28, in Socket.<anonymous>
    connection._protocol.write(data);
  File "/opt/iobroker/node_modules/mysql/lib/Connection.js", line 526, col 10, in Socket.<anonymous>
    fn.apply(this, arguments);
  File "events.js", line 198, col 13, in Socket.emit
  File "domain.js", line 448, col 20, in Socket.EventEmitter.emit
  File "    --------------------", in null.<anonymous>
  File "/opt/iobroker/node_modules/mysql/lib/protocol/Protocol.js", line 144, col 48, in Protocol._enqueue
    sequence._callSite = sequence._callSite || new Error();
  File "/opt/iobroker/node_modules/mysql/lib/protocol/Protocol.js", line 91, col 23, in Protocol.quit
    var sequence = this._enqueue(new Sequences.Quit(options, callback));
  File "/opt/iobroker/node_modules/mysql/lib/Connection.js", line 239, col 18, in Connection.end
    this._protocol.quit(opts, wrapCallbackInDomain(this, cb));
  File "/opt/iobroker/node_modules/sql-client/lib/connection-factory.js", line 22, col 20, in MySQLConnectionFactory.close_connection
    connection.end();
  File "/opt/iobroker/node_modules/sql-client/lib/sql-client.js", line 51, col 29, in SQLClient.disconnect
    return this.factory.close_connection(this.connection, (err) => {
  File "/opt/iobroker/node_modules/sql-client/lib/sql-client-pool.js", line 42, col 25, in MySQLClientPool.destroy
    return client.disconnect(callback);
  File "/opt/iobroker/node_modules/sql-client/lib/sql-client-pool.js", line 234, col 31, in passivate
    return this.destroy(client, callback);
  File "/opt/iobroker/node_modules/sql-client/lib/sql-client-pool.js", line 37, col 16, in MySQLClientPool.passivate
    return callback(null, client);
  File "/opt/iobroker/node_modules/sql-client/lib/sql-client-pool.js", line 227, col 25, in MySQLClientPool.return
    return this.passivate(client, (err, client) => {
  File "/opt/iobroker/node_modules/iobroker.sql/main.js", line 246, col 43, in returnClientToPool
    return clientPool && clientPool.return(client);

Which server are you using this for?

To be honest I have no idea. It is not an error from my system andf it was also not reported by the affected user. We just got that report via our automated Sentry crash reporting. Sorry that I can not provide more information.

I see, well, I ask because I submitted a pull requests that adds all the MySQL 8 error codes to this library, but MariaDB has their own extended codes that MySQL itself does not have. So I'm wondering if this issue can be closed when my PR is approved or if it should stay open until MariaDB's codes are (eventually) added, or if it was just some custom MySQL server entirely.


Reproducing the results for ETTh2

Hello,
Thanks a lot for the publishing your results and code, I enjoyed reading the paper.
While trying to reproduce the paper results, the output was way off especially for the ETTh2 dataset. (Ran it with the same configuration in the Colab notebook)

testing : informer_ETTh2_ftM_sl96_ll48_pl24_dm512_nh8_el3_dl2_df512_atprob_ebtimeF_dtTrue_exp_0<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
test 2857
test shape: (89, 32, 24, 7) (89, 32, 24, 7)
test shape: (2848, 24, 7) (2848, 24, 7)
mse:0.8689931831590327, mae:0.7622690107174594

Could you please let me know if you use a different hyper parameter? or what am i doing wrong .
Thanks in advance.
Regards,
Kiran

Hi,
ETTh2 has different characteristics compared to ETTh1, so you have to change hyper parameters.
You can try seq_len=48; label_len=24; pred_len=24; d_ff=2048; e_layer=2; d_layer=1 to get ETTh2 experiment prediction results, my results are:
>>>>>>>testing : informer_ETTh2_ftM_sl48_ll48_pl24_dm512_nh8_el2_dl1_df2048_atprob_fc5_ebtimeF_dtTrue_test_0<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
test 2857
test shape: (89, 32, 24, 7) (89, 32, 24, 7)
test shape: (2848, 24, 7) (2848, 24, 7)
mse:0.49008972642133736, mae:0.5386070956473465

>>>>>>>testing : informer_ETTh2_ftM_sl48_ll48_pl24_dm512_nh8_el2_dl1_df2048_atprob_fc5_ebtimeF_dtTrue_test_1<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
test 2857
test shape: (89, 32, 24, 7) (89, 32, 24, 7)
test shape: (2848, 24, 7) (2848, 24, 7)
mse:0.5389239759632527, mae:0.5776364176629023

We performed many different combinations of seq_len, label_len, e_layers and d_layers in the experiments (the values of sequence length are all multiples of the 24), and you can also try for data with different characteristics.
By the way, our current code has updated the data scaling way compared with the paper on arxiv, so we will update the experiment results later.
Thanks for your attention to our work.

Suppose there is no more discussion, and I will close this issue now.

Hi Again,
Thanks a lot for the quick response. Would it be possible for you to provide the hyper parameters for all the datasets used in the univariate and multivariate setting (for each sequence length) so that I could do a comparison. Sorry for the delay in response.
Additionally, I am getting much better results with the ETTm1 dataset while using the hyper parameters from ETTh1 dataset.

MSE
MAE

0.1510243027
0.2733270268

0.1615392239
0.283628478

Could you please confirm if these values makes sense? or am I missing something?
Thanks again.

Hi,
It seems that we named ETTm1 and ETTm2 files incorrectly in ETDataset , and we will fix this error immediately.
And we will update two new experimental results tables (due to changes of data scaling) at March 20th, and we will also provide the hyperparameter settings for each experiment at that time.
Please look forward to it. Thanks!

Hi,
ETTh2 has different characteristics compared to ETTh1, so you have to change hyper parameters.
You can try seq_len=48; label_len=24; pred_len=24; d_ff=2048; e_layer=2; d_layer=1 to get ETTh2 experiment prediction results, my results are:
>>>>>>>testing : informer_ETTh2_ftM_sl48_ll48_pl24_dm512_nh8_el2_dl1_df2048_atprob_fc5_ebtimeF_dtTrue_test_0<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
test 2857
test shape: (89, 32, 24, 7) (89, 32, 24, 7)
test shape: (2848, 24, 7) (2848, 24, 7)
mse:0.49008972642133736, mae:0.5386070956473465

>>>>>>>testing : informer_ETTh2_ftM_sl48_ll48_pl24_dm512_nh8_el2_dl1_df2048_atprob_fc5_ebtimeF_dtTrue_test_1<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
test 2857
test shape: (89, 32, 24, 7) (89, 32, 24, 7)
test shape: (2848, 24, 7) (2848, 24, 7)
mse:0.5389239759632527, mae:0.5776364176629023

We performed many different combinations of seq_len, label_len, e_layers and d_layers in the experiments (the values of sequence length are all multiples of the 24), and you can also try for data with different characteristics.
By the way, our current code has updated the data scaling way compared with the paper on arxiv, so we will update the experiment results later.
Thanks for your attention to our work.

Hi,
I have changed the code for ETTh2 according to this, my results are:

testing : informer_ETTh2_ftM_sl48_ll24_pl24_dm512_nh8_el2_dl1_df2048_atprob_ebtimeF_dtTrue_exp_0<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
test 2857
test shape: (89, 32, 24, 7) (89, 32, 24, 7)
test shape: (2848, 24, 7) (2848, 24, 7)
mse:0.5467308275443579, mae:0.574276831705922

However, the prediction figure does not match. Do I have something missing? Thank you.

Hi
I just checked the past results and the figure I draw  looks like this:

The results in the table just the mse and mae calculated by prediction and groundtruth, which does not consider the result of drawing. But according to our observations in the experiment, LSTM can also obtain very small mse and mae. But you will get a straight line when you draw its result of a longer sequence prediction.
The figure in our appendix is a sample selected from results of ETTm1 (actually it is ETTm2, we made a mistake in the naming of the file...). I checked a past result, but I’m not sure if this location of the sample is correct, and the figure looks like this:

You can also try to experiment with the parameters shown in the 'path'.
Thanks!

@cookieminions Hi, Thank you for your answer. For Etth2, the pred and true are still not matching well after I changed all the parameter according to yours, is there anything else I can change to improve the results. I also couldn't find the true figure in the Etth2 data set. Is the first prediction point start at (8569(train)+2857(val)+500(i)+48(input_len)+24(label_len))th points?

Thank you very much.

Hi,

The groundtruth used in the figure is scaled.
The first prediction point starts at (8569(train)+2857(val)+500(i))th point, because the encoder input starts at (8569(train)+2857(val)+500(i)-48(input_len))th point and decoder start token starts at (8569(train)+2857(val)+500(i)-24(label_len))th point.
My suggestion about prediction figure is that maybe you can try to use --lradj 'type2' and use larger seq_len and e_layers. Another suggestion is that you can try to experiment with a larger pred_len, such as pred_len=48, Informer will perform better than other methods in long sequence prediction. I just draw some figures using a past experiments results:

However, we only calculated the mse and mae of the model after we get the ETTh2 results, and did not do more detailed adjustments for the drawing. We also pay attention to this(especially the drawing of short sequence prediction) and will do further experiments and improvements in the future work.

Thank you very much for your attention and help to our work.
Wish you all the best with your work and research!

Hi @cookieminions
Sorry for asking so many questions, but I still couldn't get similar result as yours.
In my code, It is informer_ETTh2_ftM_sl168_ll48_pl48_dm512_nh8_el3_dl2_df512_atprob_ebfixed_dtTrue_exp_0
where your code is
informer_ETTh2_ftM_sl168_ll48_pl48_dm512_nh8_el3_dl2_df512_atprob_ebfixed_testlr1_1
How could I change to testlr1_1.

I am unsure about the testing procedure, if I understand testing part correctly, each 168(input_len)+48(label len)+48(pred_len)=264 points is one section and then repeat for the next section (264 points) for prediction. So the 48 prediction points are using model trained from previous training and validation data and then depend on 168(input len) and 48(label len) to make prediction (24 points).
Could you please also check the code whether there are any code need to be changed?
Sorry for the inconvenience. Thank you so much for your help!
Copy of Informer.ipynb - Colaboratory.pdf

Hi,
I make a graph to help explain how to get test samples from test series, hope it will be helpful:

For example, the first sample is [0, 168](seq_len)+[168-48, 168](label_len)+[168, 168+48](pred_len), the second sample is [1, 169](seq_len)+[169-48, 169](label_len)+[169, 169+48](pred_len). The label_len sequence is a part of seq_len sequence.
As for the figure, testlr1 is just the description of the experiment and the results is also obtained a long time ago (before the data scaling method change https://github.com/zhouhaoyi/Informer2020/issues/41). And I think that due to the generative decoder,  it's normal to get different figures in different repeated experiments with same parameters. So maybe you can try different datasets, prediction lengths(with different parameters) or repeated experiments, and it does not have to be the same as my figure.
And hope Informer can be helpful to your work in the use of other specific tasks.

Hi,
I make a graph to help explain how to get test samples from test series, hope it will be helpful:

For example, the first sample is [0, 168](seq_len)+[168-48, 168](label_len)+[168, 168+48](pred_len), the second sample is [1, 169](seq_len)+[169-48, 169](label_len)+[169, 169+48](pred_len). The label_len sequence is a part of seq_len sequence.
As for the figure, testlr1 is just the description of the experiment and the results is also obtained a long time ago (before the data scaling method change #41). And I think that due to the generative decoder, it's normal to get different figures in different repeated experiments with same parameters. So maybe you can try different datasets, prediction lengths(with different parameters) or repeated experiments, and it does not have to be the same as my figure.
And hope Informer can be helpful to your work in the use of other specific tasks.

Thank you for your kindness and help! I really appreciate your help.

You are welcome. Thanks for your attention to our work.
Wish you all the best with your work and research!

Hi,

The groundtruth used in the figure is scaled.
The first prediction point starts at (8569(train)+2857(val)+500(i))th point, because the encoder input starts at (8569(train)+2857(val)+500(i)-48(input_len))th point and decoder start token starts at (8569(train)+2857(val)+500(i)-24(label_len))th point.
My suggestion about prediction figure is that maybe you can try to use --lradj 'type2' and use larger seq_len and e_layers. Another suggestion is that you can try to experiment with a larger pred_len, such as pred_len=48, Informer will perform better than other methods in long sequence prediction. I just draw some figures using a past experiments results:

However, we only calculated the mse and mae of the model after we get the ETTh2 results, and did not do more detailed adjustments for the drawing. We also pay attention to this(especially the drawing of short sequence prediction) and will do further experiments and improvements in the future work.

Thank you very much for your attention and help to our work.
Wish you all the best with your work and research!

Hi @cookieminions,
When I try to find where is true points are in the dataset. I just notice the first prediction actually start at 8569(train)+2857(val)+500(i)+48+48 th point. Where the input len is 48, label len is 24, pred len is 24.
I don't understand why there is 2*input len=96 points before the prediction.
thank you.

I think maybe your confusion can be explained by code, in code data/data_loader.py line 48-51 and line 78-79 and line 95-96:
# 48-51
border1s = [0, 12*30*24 - self.seq_len, 12*30*24+4*30*24 - self.seq_len]
border2s = [12*30*24, 12*30*24+4*30*24, 12*30*24+8*30*24]
border1 = border1s[self.set_type]
border2 = border2s[self.set_type]

#78-79
self.data_x = data[border1:border2]
self.data_y = data[border1:border2]

# 95-96
def __len__(self):
    return len(self.data_x) - self.seq_len- self.pred_len + 1

# test flag:
# data_x: [12*30*24+4*30*24-self.seq_len: 12*30*24+8*30*24]
# data_x: [train_len+val_len-seq_len: train_len+val_len+test_len]

So the length of data_x in test_dataset is test_len+seq_len, and the size of test_dataset is in line 96: return len(self.data_x) - self.seq_len- self.pred_len + 1, which means test_dataset size has nothing to do with seq_len (len(data_x)=test_len+seq_len, so test_dataset size is test_len+seq_len-seq_len-pred_len+1 = test_len-pred_len+1). So you can adjust seq_len freely in the experiment.
When you load the first sample of test_dataset, you will get an encoder input with length seq_len and a groundtruth sequence with length pred_len. No matter how long your seq_len is, the first sample starts from the same place.

I also still can't reproduce the level of accuracy as reported in the paper in univariate mode.

I also still can't reproduce the level of accuracy as reported in the paper in univariate mode.

Don’t worry, we will update the experimental results and provide experimental parameters in a few days.

I think maybe your confusion can be explained by code, in code data/data_loader.py line 48-51 and line 78-79 and line 95-96:
# 48-51
border1s = [0, 12*30*24 - self.seq_len, 12*30*24+4*30*24 - self.seq_len]
border2s = [12*30*24, 12*30*24+4*30*24, 12*30*24+8*30*24]
border1 = border1s[self.set_type]
border2 = border2s[self.set_type]

#78-79
self.data_x = data[border1:border2]
self.data_y = data[border1:border2]

# 95-96
def __len__(self):
    return len(self.data_x) - self.seq_len- self.pred_len + 1

# test flag:
# data_x: [12*30*24+4*30*24-self.seq_len: 12*30*24+8*30*24]
# data_x: [train_len+val_len-seq_len: train_len+val_len+test_len]
# test_len = 4*30*24

The length of data_x in test_dataset is test_len+seq_len, and the size of test_dataset is in line 96: len(self.data_x) - self.seq_len- self.pred_len + 1, which means test_dataset size has nothing to do with seq_len (len(data_x)=test_len+seq_len, so test_dataset size is test_len+seq_len-seq_len-pred_len+1 = test_len-pred_len+1). So you can adjust seq_len freely in the experiment.
When you load the first sample of test_dataset, you will get an encoder input with length seq_len and a groundtruth sequence with length pred_len. No matter how long your seq_len is, the first sample starts from the same place.

Hi @cookieminions,
Thank you for your explanation, I am not sure I understand correctly.

When I run the code, I can see the size of Train, Validation and Test data.

start training : informer_ETTh1_ftM_sl96_ll48_pl24_dm512_nh8_el3_dl2_df512_atprob_ebtimeF_dtTrue_exp_0>>>>>>>>>>>>>>>>>>>>>>>>>>
train 8521
val 2857
test 2857

In the script, I can see
border1s = [0, 123024 - self.seq_len, 123024+43024 - self.seq_len]
border2s = [123024, 123024+43024, 123024+83024]
seq_len is 96,
border1s=[0,8544,11424]
border2s=[8640,11520,14400]
Therefore,
Train=8640,
Val=2976
Test=2976
There is 24 points different to the code output.

Yes, your understand is correct.
The size of Train, Validation and Test data you see when running data is the size of dataloader. However, in code exp/exp_informer.py line 85-90, we used drop_last=True, so the dataloader will drop the last incomplete batch (it will not affect the test samples because we use the same batch_size for all methods and the test dataloader does not be shuffled).
if flag == 'test':
    shuffle_flag = False; drop_last = True; batch_size = args.batch_size
else:
    shuffle_flag = True; drop_last = True; batch_size = args.batch_size

data_loader = DataLoader(
            data_set,
            batch_size=batch_size,
            shuffle=shuffle_flag,
            num_workers=args.num_workers,
            drop_last=drop_last)

Yes, your understand is correct.
But the size of Train, Validation and Test data you see when running data is the size of dataloader. However, in code exp/exp_informer.py line 85-90, we used drop_last=True, so the dataloader will drop the last incomplete batch (it will not affect the test samples because we use the same batch_size for all methods and the test dataloader does not be shuffled).
So the length of data_x is not the same with the length of dataloader when running the code.
if flag == 'test':
    shuffle_flag = False; drop_last = True; batch_size = args.batch_size
else:
    shuffle_flag = True; drop_last = True; batch_size = args.batch_size

data_loader = DataLoader(
            data_set,
            batch_size=batch_size,
            shuffle=shuffle_flag,
            num_workers=args.num_workers,
            drop_last=drop_last)

Great! Thank you very much.

Hi @cookieminions
I have few more questions, hope you could help me.

Is the first prediction uses seq_len in validation set?
When training and validation is finished, is it possible to add new data and make fast prediction without retrain the model.
My data is ranged from -3 to 3, which does not need to be normalised. If I don't want my data to be normalised and scaled, do I just go to dataloader.py and change scale: True to False? Will it influence on softmax and the prediction result?
Thank you very much.

Yes, data_x in test_set uses seq_len in vali_set.
Yes, it's okay to use model to predict new data without test.
You just need change the scale to False when you initialize the dataset and the data will not be scaled.

Thank you, also is it possible to output confidence interval or prediction interval?

The target of Informer is 'point prediction', so our code does not provide the function of prediction confidence interval now, but it is possible. Maybe you can refer to DeepAR to modify the last layer of the model and loss function in experiment.

Yes, data_x in test_set uses seq_len in vali_set.
Yes, it's okay to use model to predict new data without test.
You just need change the scale to False when you initialize the dataset and the data will not be scaled.

Hi @cookieminions,
I don't understand why is data_x in test_set uses seq_len in vali_set. Should we use the new data from test set as seq_len, so there won't leak any information for prediction.
Thank you.

Yes, data_x in test_set uses seq_len in vali_set.
Yes, it's okay to use model to predict new data without test.
You just need change the scale to False when you initialize the dataset and the data will not be scaled.

Hi @cookieminions,
I don't understand why is data_x in test_set uses seq_len in vali_set. Should we use the new data from test set as seq_len, so there won't leak any information for prediction.
Thank you.

Hi,
If using new data from test set as seq_len, different seq_len will cause different number of prediction samples because the length of test set series is fixed. But using seq_len in vali set, the data in vali set is just used as encoder's input and not used as prediction series, so it will not influence the test set samples.


[Share] folder structure doesn't appear in Shared with me if to share folder

EPAM AI DIAL chat version
0.7.0
What happened?

By user1 create the following folder structure
Root folder -> Stem folder -> Leaf folder
Move corresponding prompts to each folder
By user1 share the link to Stem folder
By user2 click on the link -> shared folder structure prompt does not appear in "Shared with me" -> bug
By user2 refresh the browser -> shared folder appears -> ok

Note: there is the same issue if to share simple folder with a chat, when there are no any more folder levels
Expected result: on click on the link the first available prompt should be opened automatically; root folder should appear in "Shared with me" automatically
Confidential information

[X] I confirm that do not share any confidential information

Discovered defect #1250 1250

Verified on STG, no more issues found


Add Contact Us section

I can add contact us section required at the bottom of the page. Please assign this issue to me @Pranavv213

i can work on it plz assign this to me


Poc/gs font

Official PR here or https://github.com/gluestack/gluestack-ui-flutter/pull/214.
This got a bit messy as was a POC.


Minor changelog edits

Specify that all changes are for data id 10 (AMPL/USD).

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 6.117%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
118

Relevant Lines:
1929

💛  - Coveralls


haxm much slower than kvm?

PC with haxm:  i7 6820HQ, 16 GB RAM, NVMe SSD - Windows 7, haxm 7.1, qemu 2.11 - 64 bit
PC with kvm: i7-5500U, 16 GB RAM, SATA SSH - Ubuntu 18.04, kernel 4.15, qemu 2.11 - 64 bit
A simple Ubuntu 16.04 installation lasts 16 minutes on the haxm-PC, but only 7 minutes on the kvm-PC...
Is there a technical reason for that or more a bug / inefficiency somewhere?

Recent patches have changed the instruction emulator (#42) and reduced the number of feature checks while load/saving states while entering/exiting the VM (#63). Although the emulator won't probably improve performance, the feature check patches most certainly will.
Out of sheer curiosity: If you still have the same hardware around, and building+installing the drivers is easy for you, could you rerun the tests? Just to see if we are below 16 minutes now.

@raphaelning We could really use some set of microkernels to do benchmarks of HAXM. I assume the sources of performance penalties are:

VMX exit handlers.
QEMU exit handlers.
Context switches: both VM-HAXM switches and HAXM-QEMU switches.

These three are always triggered due to a VM-exit event. So we could have a set of microkernels, each repeatedly triggering a VM-exit with a specific exit reasons, and comparing average execution time against other hypervisors.

Sadly I've no suitable build environment :-( But I've still access to the same hardware, so if you give me the binaries I could test them.

Thanks for your reply. The driver binary based on the latest code has been provided as below. You may follow the steps to test the performance.

Download HAXM driver and HaxmLoader, then extract them in a folder;
Run 'Command Prompt (Admin)' on Windows 7 64-bit and execute following commands:

sc stop IntelHaxm
HaxmLoader.exe -u
HaxmLoader.exe -i IntelHaxm.sys

@mifritscher We really appreciate your time to run these tests. Note that the test driver installed by HaxmLoader.exe is "temporary" and will disappear when you reboot the machine. You can also manually unload it:
HaxmLoader.exe -u
and then re-enable the original/official driver:
sc start intelhaxm
@AlexAltea Thanks for the inspiration. I just had a look at the kvm-unit-tests project and found this:
https://github.com/rhdrjones/kvm-unit-tests/blob/master/x86/vmexit.c
It seems to implement the same idea, except that the same microkernel image can be used to test all VM exit reasons. Although we can't reuse the code, we can definitely learn a lot from its framework, which also enables writing other microkernel-based unit tests that can be run in QEMU, e.g.:
https://github.com/rhdrjones/kvm-unit-tests/blob/master/x86/emulator.c

Ok, I exectued the testcase "install 16.04 - in a fully automated environment (via preseed)".
haxm 7.1 on the  i7 6820HQ machine: 12:25:13 until 12:37:36 (12:23)
new test version on the same machine: 12:42:00 until 12:53:18 (11:18)
Big problem: The system seemed to work (loaded the preseed.conf, exited after reasonable time, the size of the image and the extracted files are ok as well), but no graphical output after "loading from rom".
Command line:
qemu -m 1024 -smp 2 -rtc base=utc -drive file=temp.vmdk,if=virtio,detect-zeroes=unmap,discard=unmap -drive file=fat:rw:fat-type=32:label=kernel:output.dir,format=raw,if=virtio -vga std virtio-net,netdev=natted -netdev user,id=natted -kernel linux_initrd/linux -initrd linux_initrd/initrd.gz -append "nofb fb=false pti=off interface=auto auto=true priority=critical preseed/url=http://<IP_ADDRESS>:8000/preseed.cfg" -nodefaults

haxm 7.1 on the i7 6820HQ machine: 12:25:13 until 12:37:36 (12:23)
new test version on the same machine: 12:42:00 until 12:53:18 (11:18)

Great to see a near 10% performance improvement. :-)

Big problem: The system seemed to work (loaded the preseed.conf, exited after reasonable time, the size of the image and the extracted files are ok as well), but no graphical output after "loading from rom".

That's bad news. I'll look into it later as there's a non-zero chance that my emulator might have broken something: Just recently I found a movzx bug, I'll submit a patch soon enough after finishing review of https://github.com/AlexAltea/haxm/commit/699e89052f709b2d4ea6b898997e1cad712e27fd.

@raphaelning Oh, I totally forgot about kvm-unit-tests, I'll definitely run some benchmarking tests with it. Thank you!

If you update the binaries I would be keen on retesting ;-)

Some fresh data (Setup of Ubuntu 18.04, additonal cleanup, additional creation of a squashfs-image):
kvm on the Broadwell machine: 13:46 <- squashfs alone took 3:54
haxm (7.3.2) on the Skylake machine: 17:04 <-squashfs alone took 3:43
So it seems haxm got faster over the last months :-)

Thanks, that's good to know! We (@junxiaoc for the most part) have identified some inefficient VMCS read/write logic that slows down the VM exit code path. After we fix that, hopefully you'll see even better performance data.

Using Qemu 4.0.0 and haxm 7.5.1, it took 19:38, squashfs took 7:47. Note: This was done with only 1 guest CPU, as SMP 2 crashes early (see #205).


Add development instructions

These are the steps @hobozhang and I did to get this set up locally.

This is great! Other than that one comment, 👍


Possible issue trying to utilize IPv4 and IPv6 with <IP_ADDRESS> and :: at same time

Hello,
I am trying to use sslsplit on a client that has both IPv4 and IPv6 capabilities connected to an upstream MITM system bridged interface that also has both IPv4 and IPv6 capabilities.
On the client system if I try to reach sites (e.g. with curl) that are IPv6 only or sites that are only IPv4, it would seem that I would need to be able to do the following:
sudo sslsplit -D -k certauth.key -c ca.crt ssl :: 8443 tcp :: 8080 ssl <IP_ADDRESS> 8443 tcp <IP_ADDRESS> 8080
Unfortunately this does not appear to work as sslsplit references "Address already in use". If I take the command and test IPv4 and IPv6 individually:
sudo sslsplit -D -k certauth.key -c ca.crt ssl :: 8443 tcp :: 8080 will work for sites that handle both IPv6 or are only IPv6
and
sudo sslsplit -D -k certauth.key -c ca.crt ssl <IP_ADDRESS> 8443 tcp <IP_ADDRESS> 8080 will work for sites that handle IPv4
Should the first sslsplit command I pasted above work, and if not what would be your recommendation to accomplish this?
Thank you
Requested details below:
-   Output of sslsplit -V
SSLsplit 0.5.5 (built 2019-08-31)
Copyright (c) 2009-2019, Daniel Roethlisberger<EMAIL_ADDRESS>https://www.roe.ch/SSLsplit
Build info: V:FILE HDIFF:0 N:83c4edf
Features: -DHAVE_NETFILTER
NAT engines: netfilter* tproxy
netfilter: IP_TRANSPARENT IP6T_SO_ORIGINAL_DST
Local process info support: no
compiled against OpenSSL 1.1.1c  28 May 2019 (1010103f)
rtlinked against OpenSSL 1.1.1f  31 Mar 2020 (1010106f)
OpenSSL has support for TLS extensions
TLS Server Name Indication (SNI) supported
OpenSSL is thread-safe with THREADID
OpenSSL has engine support
Using SSL_MODE_RELEASE_BUFFERS
SSL/TLS protocol availability: tls10 tls11 tls12
SSL/TLS algorithm availability: !SHA0 RSA DSA ECDSA DH ECDH EC
OpenSSL option availability: SSL_OP_NO_COMPRESSION SSL_OP_NO_TICKET SSL_OP_ALLOW_UNSAFE_LEGACY_RENEGOTIATION SSL_OP_DONT_INSERT_EMPTY_FRAGMENTS SSL_OP_NO_SESSION_RESUMPTION_ON_RENEGOTIATION SSL_OP_TLS_ROLLBACK_BUG
compiled against libevent 2.1.11-stable
rtlinked against libevent 2.1.11-stable
compiled against libnet 1.1.6
rtlinked against libnet 1.1.6
compiled against libpcap n/a
rtlinked against libpcap 1.9.1 (with TPACKET_V3)
1 CPU cores detected
-   Output of uname -a
Linux utla02 5.4.0-42-generic #46-Ubuntu SMP Fri Jul 10 00:24:02 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
-   Exact command line arguments used to run sslsplit
sudo sslsplit -D -k certauth.key -c ca.crt ssl :: 8443 tcp :: 8080 ssl <IP_ADDRESS> 8443 tcp <IP_ADDRESS> 8080
-   Relevant part of debug mode (-D) output, if applicable
Generated 2048 bit RSA key for leaf certs.
SSL/TLS protocol: negotiate
proxyspecs:

[<IP_ADDRESS>]:8080 tcp netfilter
[<IP_ADDRESS>]:8443 ssl netfilter
[::]:8080 tcp netfilter
[::]:8443 ssl netfilter
Loaded CA: '/C=US/ST=WA/O=CA Lab'
SSL/TLS leaf certificates taken from:
Generated on the fly
Privsep fastpath disabled
Created self-pipe [r=3,w=4]
Created chld-pipe [r=5,w=6]
Created socketpair 0 [p=7,c=8]
Created socketpair 1 [p=9,c=10]
Created socketpair 2 [p=11,c=12]
Created socketpair 3 [p=13,c=14]
Created socketpair 4 [p=15,c=16]
Created socketpair 5 [p=17,c=18]
Privsep parent pid 2338
Privsep child pid 2339
Using libevent backend 'epoll'
Event base supports: edge yes, O(1) yes, anyfd no
Received privsep req type 03 sz 9 on srvsock 7
Received privsep req type 03 sz 9 on srvsock 7
Received privsep req type 03 sz 9 on srvsock 7
Error from bind(): Address already in use
Error opening socket: Address already in use (98)
Failed to initialize proxy.
Child pid 2339 exited with status 1

-   NAT redirection rules you are using, if applicable
Copy of script I created to setup bridge and associated nft config below:
#!/bin/bash
#By Chaskel 2020-07-25 v1.0
#Note: Tested on Ubuntu 20.04 which uses nftables to replace iptables/ip6tables/ebtables/etc.
#DESCRIPTION:
#Sets system interfaces up for doing sslsplit MITM traffic observation
#Modify lines below to reflect your needs
#Be sure to execute script with sudo
#Set the interface names of the 2 NICs that traffic will traverse
NIC1=eth1
NIC2=eth2
#Set the interface name of the bridge
BR=br0
#Set the ports you would like to redirect keep the same formatting as listed below
NON_SSL="80, 5522"
SSL="443, 465, 587, 993"
#Set the redirect ports relating to the ports above
REDIR_NON_SSL=8080
REDIR_SSL=8443
######## DO NOT MODIFY LINES BELOW ########
#Create bridge interface
ip link set $NIC1 up
ip link set $NIC2 up
ip link add name $BR type bridge
ip link set $BR up
ip link set $NIC1 up
ip link set $NIC1 master $BR
ip link set $NIC2 up
ip link set $NIC2 master $BR
#Define variable with MAC of bridge interface
BR_MAC=$(cat /sys/class/net/$BR/address)
#Assign DHCP address to bridge interface
#Note: If you see a "cmp: EOD on /tmp......." message you may most likely ignore it
dhclient $BR
#Execute nftables related items
nft add table bridge table1
nft -- add chain bridge table1 chain1 { type filter hook prerouting priority -200 ; }
nft add rule bridge table1 chain1 tcp dport { $NON_SSL } meta pkttype set host ether daddr set $BR_MAC counter
nft add rule bridge table1 chain1 tcp dport { $SSL } meta pkttype set host ether daddr set $BR_MAC counter
nft add table inet table2
nft add chain inet table2 chain2 { type nat hook prerouting priority 0 ; }
nft add rule inet table2 chain2 tcp dport { $NON_SSL } counter redirect to $REDIR_NON_SSL
nft add rule inet table2 chain2 tcp dport { $SSL } counter redirect to $REDIR_SSL

As I am sure you have realized, you are trying to use the same port in two different proxyspecs, twice: 8443 and 8080. Hence, you get an "Address already in use" error, even if the one is for IPv4 and the other for IPv6. You should use different ports for each proxyspec. So I recommend the following command line:
sudo sslsplit -D -k certauth.key -c ca.crt ssl :: 8443 tcp :: 8080 ssl <IP_ADDRESS> 8444 tcp <IP_ADDRESS> 8081
But this of course is possible if you can configure your target device to listen on 8444 and 8081 for IPv4.
On an unrelated issue, sslsplit reports that you are compiling and linking sslsplit using different OpenSSL versions.

Hello,
Thank you for the quick response and guidance. I appear to have successfully accomplished what you recommended and I greatly appreciate your tip.
In case the following information is helpful:

Please note that the reason I had thought that I could use the syntax I referenced in my original message is due to the very bottom line with the example on "sslsplit -h" where it shows both <IP_ADDRESS> and ::1 with https and port 8443.
I had also tried using the IPv4 address assigned to the bridge along with the link local IPv6 (fe80) address assigned to the bridge but sslsplit did not appear to like the link local IPv6 address. It is very possible I did something incorrectly, but I thought I would mention this just in case there is in fact an issue (with the solution you provided I do nit currently need to pursue this method).
Regarding your OpenSSL comment, please note that the output I provided is from a new Ubuntu 20.04 install I performed yesterday (using mini.iso) and I installed sslsplit via apt (I did not do any manual compiling/linking).

I am also including a copy of my updated script in case this could help anyone else in the future:
#!/bin/bash
#By Chaskel 2020-07-26 1.0.1
#Note: Tested on Ubuntu 20.04 system with IPv4/IPv6 using nftables (replaces iptables/ip6tables/ebtables/etc.)
#DESCRIPTION:
#Sets system interfaces up for doing sslsplit MITM traffic observation
#Modify lines below to reflect your needs
#Be sure to execute script with sudo
#Set the interface names of the 2 NICs that traffic will traverse
NIC1=eth1
NIC2=eth2
#Set the interface name of the bridge
BR=br0
#Set the ports you would like to redirect and keep the same formatting as listed below
NON_SSL="80, 5522"
SSL="443, 465, 587, 993"
#Set the redirect ports relating to the ports above
REDIR_NON_SSL_IPV4=8080
REDIR_NON_SSL_IPV6=8081
REDIR_SSL_IPV4=8443
REDIR_SSL_IPV6=8444
######## DO NOT MODIFY BELOW THIS LINE ########
#Create bridge interface
ip link set $NIC1 up
ip link set $NIC2 up
ip link add name $BR type bridge
ip link set $BR up
ip link set $NIC1 up
ip link set $NIC1 master $BR
ip link set $NIC2 up
ip link set $NIC2 master $BR
#Define variable with MAC of bridge interface
BR_MAC=$(cat /sys/class/net/$BR/address)
#Assign DHCP address to bridge interface
#Note: If you see a "cmp: EOD on /tmp......." message you may most likely ignore it
dhclient $BR
#Execute nftables related items
nft add table bridge table1
nft -- add chain bridge table1 chain1 { type filter hook prerouting priority -200 ; }
nft add rule bridge table1 chain1 tcp dport { $NON_SSL } meta pkttype set host ether daddr set $BR_MAC counter
nft add rule bridge table1 chain1 tcp dport { $SSL } meta pkttype set host ether daddr set $BR_MAC counter
nft add table ip table2
nft add chain ip table2 chain2 { type nat hook prerouting priority 0 ; }
nft add rule ip table2 chain2 tcp dport { $NON_SSL } counter redirect to $REDIR_NON_SSL_IPV4
nft add rule ip table2 chain2 tcp dport { $SSL } counter redirect to $REDIR_SSL_IPV4
nft add table ip6 table3
nft add chain ip6 table3 chain3 { type nat hook prerouting priority 0 ; }
nft add rule ip6 table3 chain3 tcp dport { $NON_SSL } counter redirect to $REDIR_NON_SSL_IPV6
nft add rule ip6 table3 chain3 tcp dport { $SSL } counter redirect to $REDIR_SSL_IPV6

Thanks @Chaskel for detailed explanations.
You are right about the usage help, there is an example with loopback addresses for both IPv4 and IPv6, which by the way works without any issues. Afaik, you can do that with individual IP addresses, but if you use "unspecified" addresses, as they call it, e.g. <IP_ADDRESS> and ::, I read that it does not work. Quoting from here:

"Servers can either listen on a specific IP address/port combination or can listen on a port for all addresses. The latter is specified by specifying a listen address of <IP_ADDRESS> for IPv4 or :: for ipv6. If a server is listening on a port for all addresses then other servers can't listen on that port for a specific address."

I don't know the reason behind this difference. Sorry for the confusion.
You are right about the version issue of OpenSSL. The same issue exists on Linux Mint 20 too, quite naturally as it uses Ubuntu packages. I don't see any openssl updates yet, so I find it a bit strange that they have compiled the sslsplit package on an older system than the one they have released for. I hope the differences between 1.1.1c and 1.1.1f do not cause any issues with sslsplit.

No problem @sonertari and I appreciate the help.
Thank you again for your support and solution.


Public folder created outside root directory

I've been getting this odd issue lately with the Public folder being created outside of the root folder structure. I've done a few different tests with it and can't really seem to point my finger on the cause. It happens when I run 'roots watch' and I'm not sure if it's because of certain project names or what.
For instance: I ran a 'roots new mobile' and this issue occured. However, running 'roots new mobileportfolio' worked and functioned as normal.
Currently running MacOS Sierra 10.12.1 on the Late 2016 Touchbar MBP 13".

Hi @kylefraser - I've not used Roots in some time, but I do remember having a similar issue to this at one point - the source of the problem for me was a rogue package.json one level up from my roots project folder. To illustrate:
- projects/
---package.json <-- This caused the issue for me
---roots_project/
------app.coffee


Support kubectl apply steps

Fixes https://github.com/GoogleCloudPlatform/opentelemetry-demo/issues/24
Update kubernetes/opentelemetry-demo.yaml file to match our Helm and docker-compose deployments. This file is generated in the upstream demo repo, but since we aren't generating it we can edit it directly and manually handle conflicts for now.
This also makes some updates to the Workload Identity steps to align the Helm/Kubectl install. By default, kubectl apply will install the collector in the otel-demo namespace while Helm installs in default. Instead of making more changes to the kubectl manifests or having 2 sets of instructions I just updated the commands to Helm install in the otel-demo namespace too.
In this change, I noticed that some of the processors that were removed in the Helm chart (see https://github.com/GoogleCloudPlatform/opentelemetry-demo/issues/23) were still present in this file. I don't know if that's because of catching this in the middle of an upstream release, or if upstream just didn't update that file, but I manually tested this and it still deploys. When we have integration testing that will have to check this when we rebase for the next release.
Tested both kubectl apply and Helm install manually.

Will this make rebases on upstream really hard?  IIRC, the plan was to try and automate those to some extent.

Will this make rebases on upstream really hard? IIRC, the plan was to try and automate those to some extent.

I don't think so, this will be one file that we'll have to resolve minimal conflicts with. Automating that will be tough, so maybe we could look at a different way to do this (like an entirely separate config file that gets kubectl apply'd to overwrite the default one?)


Fix promotion validation - pre-existence of paths for remote repos

When the no-pre-existing-path.groovy was performed for a remote repo,
e.g. check if a GAV is not already present in Maven Central, it
successfully found the artifact only if it was cached. For artifacts
not in cache it acted like it was not present, because tools.getTransfer
does not retrieve an artifact from a remote repo and returns a not-null
value only for artifacts present in local storage.
One option was to use exists, which is what I chose, but bear in mind
that this way we are loosing the info about source repo, e.g. instead of
"... is already available in: remote:central" we get "... is already
available in: group:public". To preserve the info about the source repo
we would need to use retrieve, which returns Transfer containing the
necessary info, but retrieving the artifacts could slow the validation
down even more and cause more timeouts. Right now we are aiming for
promotion rules as lightweight as possible before we run them in an
async way.
(Cherry-pick of #508 to master)

retest this please


Do not fail on openID connector

I ran into an account that has OpenID connectors. Until I understand more, I'd rather have it not fail...

Looking at the code around this I realized I'm silently ignoring cognito, which isn't ideal, so I ticketed this for myself (https://github.com/duo-labs/cloudmapper/issues/660).  I should at least print a warning for this.  I'm going to merge your PR in because it's doing the same thing as I was doing previously, but I'm going to print a warning about this unknown thing at some point.


pre-commit: PRantoniofrighetto/llvm-project/pull/1

Link: https://github.com/antoniofrighetto/llvm-project/pull/1
Requested by: @antoniofrighetto

/add-label reviewed

/close


Add support for PostgreSQL

Can you add support for PostgreSQL inside PHP container, please?

I've added php-pgsql and php-pdo_pgsql both for PHP 5.6 and 7.0.

Thanks a lot. Can you also add postgresql-client (as we have client for MariaDB)?

Done.


Precise timestamp

Hi, nice work.
It seems that the timestamps are based on ctc greedy search.
Have you tested the accuracy? From my own experience, timestamps from CTC peaks are not so accurate.
Thanks.

yeah, your are right. The timestamps of SenseVoice is terrible.


getting docker usage output when running kitchen command

Hi,
I don't know when it started but now there is an annoying bug,
Whenever I run kitchen with a subcommand, I get the docker usage in the output.

master now redirecting both stderr and stdout to /dev/null.


Issue #54: Some new palettes

Closes #54
Looking for color change suggestions before we merge. Also had to modify the request index table column lengths to get it to play nice with a non white background

In the event that this is picked up by someone else later down the road, I pulled the actual color codes into variables, so changing the colors should be as simple as just changing the variables. The UI may look a little different by then, so you may need to go and tweak the component overrides.


[6.x] Composer fix monolog version range

Replaces deprecated composer's logical Or from "|" to "||", because official composer's documentation claims that only that should be used.
https://getcomposer.org/doc/articles/versions.md#version-range
A double pipe (||) will be treated as a logical OR
Double pipe is Or operator, but single pipe is logical Xor.
Official issue on composer github

Please update all composer.json files in the repo that ref monolog.

Ok, i will do it some later, thank you

I took the liberty to revert this. The single pipe is still supported and won't be removed as @stof mentions here: https://github.com/composer/composer/issues/6755
At the moment all of the Laravel libraries use the single pipe formatting and to stay consistent it's best that we just leave things be. There's no harm in keeping the current formatting.
Also reverted https://github.com/laravel/framework/pull/30637

I don't see why we should revert this and remain on a deprecated solution. IMO we should do the exact opposite and fix this in all composer.json files.

The old syntax gives maximum compatibility with old composer versions.

Well, composer versions not supporting it are not relevant anymore. They won't be able to use Packagist anymore, since they don't support the composer repositories splitting metadata into separate files (Packagist kept the older format for a while to give time to upgrade composer, but at some point, it dropped the old format which was insane for performance)

Ah, right. :P


SRI tag helper security\fallback

As Damian on the asp.net community standup noted.
It's not secure to compute the hash on the server directly from the cdn (even if you cache the result).
Because the file could become compromised at some point in time.
Another note is this > CDNs fail, but your scripts don't have to - fallback from CDN to local jQuery
I would recommend uploading the required file to the server, use that to compute the hash (saves on download time).
And generate a fallback script for when the cdn fails or becomes corrupted like so:
<script src="http://ajax.aspnetcdn.com/ajax/jquery/jquery-2.0.0.min.js"></script>
<script>
if (typeof jQuery == 'undefined') {
    document.write(unescape("%3Cscript src='/js/jquery-2.0.0.min.js' type='text/javascript'%3E%3C/script%3E"));
}
</script>

and of course you could add the SRI tag to both script elements

My initial thinking was that you could check the files at deployment time when the tag helper first runs. Then the tag helper would have calculated the hash and cached it without any expiration time, so you are good from then on.
I suppose that checking the files on every deployment is not great for the developer. So for the next iteration I will add an alternative source, basically a local file from which the SRI is calculated. Someting like this:
<script src="https://somecdn/jquery.js"
            asp-subresource-integrity
            asp-subresource-integrity-alt-src="~/js/jquery.js"></script>

In this scenario, the developer would have to check the file for validity every time they upgrade it. I'm not 100% sure if dropping the call to the CDN resource is the way to go. If we kept the call, we could compare the SRI and log an error, giving the developer some quick feedback to something they might otherwise miss if they are using script fallbacks. Perhaps this should be an option?
There is already a fallback script tag helper built in which pretty much generates the code you wrote. However, I don't like them for two reasons. The first is that they are not compatible with Content Security Policy (CSP) because they require the use of inline scripts. Secondly they don't work for dependant scripts e.g. if you have bootstrap.js which is dependant on jquery.js.

Blog Post

Looks great, I totally forgot about the fact that asp.net core already has it included (I'm currently still stuck on asp.net until I get my app rebuilt for core).
As far as I'm concerned you can close this issue.


Deprecation warning for servicePath can't be resolved in TypeScript.

Environment details

OS: Mac OS 14.4.1 (23E224)
Node.js version: v18.19.1
npm version: 10.2.4
@google-cloud/firestore version: 7.6.0

Steps to reproduce

Call v1.FirestoreClient.apiEndpoint
The console logs DeprecationWarning: Static servicePath is deprecated, please use the instance method instead.

This commit added the deprecation warning and also the new field to use: https://github.com/googleapis/nodejs-firestore/commit/d964a13818e6bc4be6547e88ffbe16be1a6ceeeb. However, it is not possible to call the apiEndpoint instance method because the TypeScript interface was not updated with the new function: https://github.com/googleapis/nodejs-firestore/blob/9da9fb031cd6eb6ee83062c339d0448f3b37903b/types/v1/firestore_client.d.ts#L111-L116
It misses both the @deprecated documentation and the new getter.

Hmm, it looks like the .d.ts files in the types subdirectory (other than firestore.d.ts) are supposed to be updated automatically by the owlbot build system; however, this appears to be broken.
Here is a timeline:

https://github.com/googleapis/nodejs-firestore/pull/1433 (March 2021) adds the .d.ts files, such as types/v1/firestore_client.d.ts and configures "autosynth" to automatically update them.
https://github.com/googleapis/nodejs-firestore/pull/1564 (July 2021) is the last autosynth PR that updates these .d.ts files.
https://github.com/googleapis/nodejs-firestore/pull/1587 (August 2021) migrates from "autosynth" to "owlbot"
The .d.ts files are never updated again by any automated tools.

So it looks like something went wrong in the migration from autosynth to owlbot. I'll look into it.

v7.9.0 releases the updated auto-gen types, which includes the @deprecated tag and new getter.


[d3d9] Optimize pure SWVP devices

Needs lots of testing.
This makes D3D9 devices, that are configured to always use software vertex processing (so not MIXED), always use the late per draw buffer upload path. We copy the vertex data that each specific draw accesses to a temporary buffer and render from that, similar to Up-draws.
This makes sense because games that use pure SWVP expect vertex processing to be synchronous which has lead to both bugs and performance problems. For example we used to run into issues when respecting NOOVERWRITE or have dozens or even hundreds of queue syncs per frame. Considering that SWVP is supposed to run on the CPU, the amount of vertices is hopefully small.
I hope this won't impact more modern or demanding games.
The game that inspires this was Phantasmat from this comment:
https://github.com/doitsujin/dxvk/issues/4263#issuecomment-2358659238
It uses a single 96,000 byte vertex buffer (POOL = DEFAULT, USAGE = WRITEONLY, FVF != 0) and writes data to it before every single draw. Ofc it also doesn't specify a lock range, so we end up uploading the entire 96 KB buffer over and over again, run out of staging memory and then stall. It is a 2D game, so with this PR we upload 4 vertices for every draw.

cc @WinterSnowfall

I can throw this at a bunch of games of course, but I think it would be useful and very helpful (since enhancing the HUD is a trend now) to also add the type of VP (based on device type and on m_isSWVP in case of Mixed) as an element to the D3D9 HUD.
The use of D3DCREATE_SOFTWARE_VERTEXPROCESSING devices is AFAIK very limited even in d3d8 and generally only used as a fallback in case HW or Mixed modes fail. A very limited set of games let you pick which to use.

This PR also properly fixes https://github.com/AlpyneDreams/d8vk/issues/179 , on which we had more or less given up in d8vk. The Supreme Commander d3d8 games can now be played with correct text rendering even without the "Nvidia workaround" configuration (which affected performance very negatively).

This PR also properly fixes AlpyneDreams#179 , on which we had more or less given up in d8vk. The Supreme Ruler d3d8 games can now be played with correct text rendering even without the "Nvidia driver workaround" configuration option (which affected performance very negatively).

Back then i give up too, but it worked fine as is on Gallium NINE. Now we know what is a problem, as NINE have SVPs optimized.
Now you got it too.

Now you know what is a problem, as NINE have SVPs optimized.

It has nothing to do with that.

Of course, could be something else as. Phantasmat regressed from yesterday.


[Snyk] Security upgrade storybook from 7.6.11 to 8.2.0

This PR was automatically created by Snyk using the credentials of a real user.![snyk-top-banner](https://github.com/andygongea/OWASP-Benchmark/assets/818805/c518c423-16fe-447e-b67f-ad5a49b5d123)
Snyk has created this PR to fix 2 vulnerabilities in the npm dependencies of this project.
Snyk changed the following file(s):

package.json
package-lock.json

Vulnerabilities that will be fixed with an upgrade:

Issue
Score

Missing Release of Resource after Effective Lifetime SNYK-JS-INFLIGHT-6095116
  141

Regular Expression Denial of Service (ReDoS) SNYK-JS-ASYNC-7414156
  49

[!IMPORTANT]

Check the changes in this PR to ensure they won't cause issues with your project.
Max score is 1000. Note that the real score may have changed since the PR was raised.
This PR was automatically created by Snyk using the credentials of a real user.

Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs.
For more information:
🧐 View latest project report
📜 Customise PR templates
🛠 Adjust project settings
📚 Read about Snyk's upgrade logic

Learn how to fix vulnerabilities with free interactive lessons:
🦉 Regular Expression Denial of Service (ReDoS)

Closed by https://github.com/equinor/videx-map/pull/104


KafkaConsumerSpec is failing due to OutOfMemoryError

When I run
sbt clean coverage teset

I see the following
[info] KafkaConsumerSpec:
[ERROR] [02/06/2015 13:35:46.414] [NIOServerCnxnFactory] Thread Thread[Thread-23,5,main] died
java.lang.OutOfMemoryError: PermGen space

I believe we need to provide these settings in project/Build.sbt
javaOptions += "-XX:MaxPermSize=512m",
javaOptions += "-Xmx2048m",

close this as it canot reproduced anymore.