HADOOP-17074 Optimise s3a Listing to be fully asynchronous.

Tested using ap-south-1 bucket. All good apart from known failures.
https://issues.apache.org/jira/browse/HADOOP-17192
https://issues.apache.org/jira/browse/HADOOP-17190

Performance result using new test:
2020-08-10 14:55:31,985 [JUnit-testMultiPagesListingPerformanceAndCorrectness] INFO  contract.ContractTestUtils (ContractTestUtils.java:end(1847)) - Duration of listing 500 files using listFiles() api with batch size of 10 including 10ms of processing time for each file: 7,394,348,570 nS 2020-08-10 14:55:42,288 [JUnit-testMultiPagesListingPerformanceAndCorrectness] INFO  contract.ContractTestUtils (ContractTestUtils.java:end(1847)) - Duration of listing 500 files using listStatus() api with batch size of 10 including 10ms of processing time for each file: 10,204,574,314 nS
CC @steveloughran @mehakmeet @bgaborg

Fixed all review comments and re ran the new test. All good.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
0m 36s
Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s
No case conflicting files found.

+1 :green_heart:
@author
0m  0s
The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s
The patch appears to include 2 new or modified test files.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
33m 37s
trunk passed

+1 :green_heart:
compile
0m 40s
trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1

+1 :green_heart:
compile
0m 34s
trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01

+1 :green_heart:
checkstyle
0m 26s
trunk passed

+1 :green_heart:
mvnsite
0m 43s
trunk passed

+1 :green_heart:
shadedclient
15m  4s
branch has no errors when building and testing our client artifacts.

+1 :green_heart:
javadoc
0m 20s
trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1

+1 :green_heart:
javadoc
0m 27s
trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01

+0 :ok:
spotbugs
1m  5s
Used deprecated FindBugs config; considering switching to SpotBugs.

+1 :green_heart:
findbugs
1m  4s
trunk passed

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
0m 35s
the patch passed

+1 :green_heart:
compile
0m 35s
the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1

+1 :green_heart:
javac
0m 35s
the patch passed

+1 :green_heart:
compile
0m 27s
the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01

+1 :green_heart:
javac
0m 27s
the patch passed

-0 :warning:
checkstyle
0m 17s
hadoop-tools/hadoop-aws: The patch generated 6 new + 12 unchanged - 0 fixed = 18 total (was 12)

+1 :green_heart:
mvnsite
0m 32s
the patch passed

+1 :green_heart:
whitespace
0m  0s
The patch has no whitespace issues.

+1 :green_heart:
shadedclient
13m 38s
patch has no errors when building and testing our client artifacts.

+1 :green_heart:
javadoc
0m 19s
the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1

+1 :green_heart:
javadoc
0m 25s
the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01

+1 :green_heart:
findbugs
1m 10s
the patch passed

_ Other Tests _

+1 :green_heart:
unit
1m 26s
hadoop-aws in the patch passed.

+1 :green_heart:
asflicense
0m 33s
The patch does not generate ASF License warnings.

75m 39s

Subsystem
Report/Notes

Docker
ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2207/2/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/2207

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle

uname
Linux b39672fe98f7 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
personality/hadoop.sh

git revision
trunk / e592ec5f8bf

Default Java
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01

checkstyle
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2207/2/artifact/out/diff-checkstyle-hadoop-tools_hadoop-aws.txt

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2207/2/testReport/

Max. process+thread count
419 (vs. ulimit of 5500)

modules
C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2207/2/console

versions
git=2.17.1 maven=3.6.0 findbugs=4.0.6

Powered by
Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

LGTM. +1 pending the changes needed to get checkstyle to be (mostly) quiet
nice bit of work here.

...LGTM, let's see what Yetus says

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
1m 17s
Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s
No case conflicting files found.

+1 :green_heart:
@author
0m  0s
The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s
The patch appears to include 2 new or modified test files.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
32m 28s
trunk passed

+1 :green_heart:
compile
0m 53s
trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1

+1 :green_heart:
compile
0m 42s
trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01

+1 :green_heart:
checkstyle
0m 30s
trunk passed

+1 :green_heart:
mvnsite
0m 49s
trunk passed

+1 :green_heart:
shadedclient
19m 27s
branch has no errors when building and testing our client artifacts.

+1 :green_heart:
javadoc
0m 20s
trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1

+1 :green_heart:
javadoc
0m 30s
trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01

+0 :ok:
spotbugs
1m 17s
Used deprecated FindBugs config; considering switching to SpotBugs.

+1 :green_heart:
findbugs
1m 15s
trunk passed

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
0m 37s
the patch passed

+1 :green_heart:
compile
0m 39s
the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1

+1 :green_heart:
javac
0m 39s
the patch passed

+1 :green_heart:
compile
0m 32s
the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01

+1 :green_heart:
javac
0m 32s
the patch passed

+1 :green_heart:
checkstyle
0m 21s
the patch passed

+1 :green_heart:
mvnsite
0m 35s
the patch passed

+1 :green_heart:
whitespace
0m  0s
The patch has no whitespace issues.

+1 :green_heart:
shadedclient
17m 44s
patch has no errors when building and testing our client artifacts.

+1 :green_heart:
javadoc
0m 19s
the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1

+1 :green_heart:
javadoc
0m 28s
the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01

+1 :green_heart:
findbugs
1m 27s
the patch passed

_ Other Tests _

+1 :green_heart:
unit
1m 44s
hadoop-aws in the patch passed.

+1 :green_heart:
asflicense
0m 35s
The patch does not generate ASF License warnings.

84m 53s

Subsystem
Report/Notes

Docker
ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2207/3/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/2207

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle

uname
Linux fb536f212292 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 17cd8a1b162

Default Java
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2207/3/testReport/

Max. process+thread count
342 (vs. ulimit of 5500)

modules
C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2207/3/console

versions
git=2.17.1 maven=3.6.0 findbugs=4.0.6

Powered by
Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.


fix nodePort for weave

When using the weave network plugin I ran into the problem that nodePort for services like the nginx ingress.
The problem is that originally cni did not care for portmappings etc. Now there is a plugin for portmappings, but weave leaves it to the k8s admin to configure that.
See https://github.com/weaveworks/weave/issues/3016

ci check this


Issue in capture and analyse - Infer

@cristianoc
I have read the article to make the infer to analyze only the modified files.
https://github.com/facebook/infer/issues/596
Can i follow the same approach for android too?
Where should the index.txt file should reside?
I followed two steps:
infer -a capture -- ./gradlew --offline assLollipopDeb
infer --reactive --changed-files-index ./index.txt -- analyze
I know there are two issues in the file which i given in index.txt. But its not working. I tried giving both relative path and absolute path of the file in index.txt. I placed index.txt inside infer-out.
Still the proc_stats_json contains:
{ "files": 0, "procedures": 0, "lines": 0 }
Can you please elaborate if any different steps are available for Android?

Can you try this on the example given in the Infer distribution.
These are the same commands you're mentioning, and they work for me.
cd infer/examples/android_hello
./gradlew clean
infer -a capture -- ./gradlew build

cat ./index.txt
app/src/main/java/infer/inferandroidexample/MainActivity.java

infer --reactive --changed-files-index ./index.txt -- analyze

After excuting the following command, still the index.txt file is empty.
cat ./index.txt app/src/main/java/infer/inferandroidexample/MainActivity.java
But index.txt should include the file path right?

Sorry for the confusion: you should edit index.txt so that it contains one line:
app/src/main/java/infer/inferandroidexample/MainActivity.java

Then give the command
infer --reactive --changed-files-index ./index.txt -- analyze

Should the file reside inside the infer-out or outside?

Outside. "infer-out" may get wiped out by infer if you run infer without --reactive.

Still not working. Still saying No Issues found.
I tried
./gradlew clean
infer -a capture -- ./gradlew build --offline assLollipopDeb
cat ./index.txt app/src/main/java/com/olacabs/customer/ui/GreenCategoryHelper.java
infer --reactive --changed-files-index ./index.txt -- analyze
Inside index.txt file, i tried both of the below path:
/Users/mohammedsafiq.sa/Documents/Projects/android/app/src/main/java/com/olacabs/customer/ui/GreenCategoryHelper.java
app/src/main/java/com/olacabs/customer/ui/GreenCategoryHelper.java
Nothing worked.

I don't think your cat command does anything. Try:
./gradlew clean
infer -a capture -- ./gradlew build --offline assLollipopDeb
echo app/src/main/java/com/olacabs/customer/ui/GreenCategoryHelper.java > index.txt
infer --reactive --changed-files-index ./index.txt -- analyze

Dint work. I am just following your command. Still not sure why it is not throwing.

Ok now its analysing. Actually its /index.txt and not ./index.txt.
It said 7 source files found and analysed. How come 7 files when i hardcoded one file path in index.txt?

@mohammedsafiq In order to analyze a file, infer has to analyze all the procedures called from that file. So if some of these procedures live in different files then they are also analyzed. That doesn't mean that all the procedures in these files were analyzed, only the ones called (transitively!) from the original file.

Looks like this is resolved, please reopen if I'm mistaken.


@xtina-starr => [Onboarding] Passes forceStep to Wizard from slug

Need to wait on https://github.com/artsy/reaction/pull/511

This looks good to me. Just had a couple of questions. Merge whenever!

👍


Have trpc as peer dependency

Is it possible to perhaps copy from how trpc-next does this https://github.com/trpc/trpc/blob/next/packages/next/package.json
I believe this will help reduce maintenance burden on always needing to keep track with versions, with only needing to upgrade when library needs it as a dev dependency and leave responsibility to userland to maintain trpc package version

peer dep in v3 and v4 now.

Thanks! This is amazing, I can setup renovate to auto update trpc on my side now


Per rule ignore with hypens are ignored

Describe the bug
deptry currently fails to ignore dependencies which have a hyphen (-) in it.
This is reliably reproducible either with the CLI and also with a pyproject.toml configuration.
To Reproduce
Add this to your pyproject.toml:
[tool.deptry.per_rule_ignores]
DEP003 = ["mypy-boto3-dynamodb"]

Run deptry . while a file has this dependency imported
Expected behavior
The dependency should be ignored.

System [please complete the following information]:

OS: Debian 11 (bullseye)
Language Version: Python 3.12
Poetry version: uv 0.1.24

The ignore rules for DEP003 should match the Python module name, not the package name. This is unfortunately wrongly reported by deptry.
Using this configuration should work as expected:
[tool.deptry.per_rule_ignores]
DEP003 = ["mypy_boto3_dynamodb"]

https://github.com/fpgmaas/deptry/pull/644 should fix the reported violation.


Makefile: allow overriding some vars via env.

Hey, me again :D
Since luarocks are a bit meh to use in nvim right now, I'd like to add the option of installing jsregexp via some command to luasnip.
My current plan for doing this is

find lua-version used via nvim (nvim -v provides some info here) and
modify INCLUDE_DIR and LDLIBS based on that.

For that, these vars would have to be overridable, which is what this PR implements.
Alternatively, the check for the available lua-version could live in this repo, which would be nicer IMO, but with that there's a weird, slightly out-of-place dependency on nvim, not sure if you want that (maybe a separate with_nvim/for_nvim-target with the check?)
WDYT? Would you be up for maintaing the cross-platform install (only linux for now I think, I don't yet know how to link agains lua on windows) or do you dislike that approach?

Hey! You can already override these variables by invoking make INCLUDE_DIR=/usr/include/luajit-2.1. We are not actually linking against liblua5.1.so and that flag should be removed from the makefile. And for the include directory it doesn't actually matter which sets of headers you use. Luajit's headers are a strict superset of lua5.1's and completely compatible (we don't use any luajit exclusives). What remains is finding the include directory of either lua5.1 or luajit.
Detecting and and finding these directories doesn't really have a place here, that's usually when you move from make to cmake (not happening) or have luarocks take care of it.

Hey! You can already override these variables by invoking make INCLUDE_DIR=/usr/include/luajit-2.1

I though so too, but that didn't work, lemme try again..
Ohh, found my mistake, I though those were set via env before make, not as an arg to it :facepalm:
Works now, thank you!

And for the include directory it doesn't actually matter which sets of headers you use. Luajit's headers are a strict superset of lua5.1's and completely compatible (we don't use any luajit exclusives).

Aha, that's good to know :+1: Then I can just glob to match either/both

(not happening)

xD

If you are going to download the sources to build them with make, you could just go ahead and also download the headers.

I hadn't thought of that... Might actually be better, some platforms ship headers and runtime-stuff separately, we'd stay compatible with those
Okay great, thanks for the advice :+1:


RuntimeError: Cannot pause_reading() when closing, triggered via FlowControlStreamReader.feed_data

Long story short
When trying to read the response to a get request from an https endpoint, I get a RuntimeError: Cannot pause_reading() when closing exception.
Expected behaviour
I expect it to read the response successfully.
Actual behaviour
  File "/usr/lib64/python3.5/site-packages/aiohttp/client_reqrep.py", line 739, in read
    self._content = yield from self.content.read()
  File "/usr/lib64/python3.5/site-packages/aiohttp/streams.py", line 507, in wrapper
    result = yield from func(self, *args, **kw)
  File "/usr/lib64/python3.5/site-packages/aiohttp/streams.py", line 562, in read
    return (yield from super().read(n))
  File "/usr/lib64/python3.5/site-packages/aiohttp/streams.py", line 265, in read
    raise self._exception
  File "/usr/lib64/python3.5/site-packages/aiohttp/parsers.py", line 190, in set_parser
    next(p)
  File "/usr/lib64/python3.5/site-packages/aiohttp/protocol.py", line 313, in __call__
    yield from self.parse_length_payload(out, buf, length)
  File "/usr/lib64/python3.5/site-packages/aiohttp/protocol.py", line 359, in parse_length_payload
    out.feed_data(chunk, len(chunk))
  File "/usr/lib64/python3.5/site-packages/aiohttp/streams.py", line 554, in feed_data
    self._stream.transport.pause_reading()
  File "/usr/lib64/python3.5/asyncio/sslproto.py", line 336, in pause_reading
    self._ssl_protocol._transport.pause_reading()
  File "/usr/lib64/python3.5/asyncio/selector_events.py", line 647, in pause_reading
    raise RuntimeError('Cannot pause_reading() when closing')
RuntimeError: Cannot pause_reading() when closing

Steps to reproduce
The relevant code is something like this, though I've only been able to reproduce it in an application which creates many sessions simultaneously:
async def ssl_fetch(url):
    with aiohttp.ClientSession(
        connector=aiohttp.TCPConnector(
        ssl_context=ssl._create_unverified_context())) as session:

        resp = await session.get(url)
        content = await resp.read()
        await resp.release()
        return content

Your environment
I have observed this bug with aiohttp versions 1.0.1 and 1.0.2 (have not tried any older versions), on Gentoo Linux with Python 3.5.2. I've only been able to reproduce it in an application which creates many sessions simultaneously. Each session has a single request directed at an https endpoint on port 5080 of Linux host running ganeti (http://docs.ganeti.org/ganeti/2.5/man/ganeti-rapi.html).
I was able to workaround the problem by overriding the ClientSession response_class parameter as follows:
class _FlowControlStreamReader(aiohttp.streams.StreamReader):
    def feed_data(self, data, size=0):
        super().feed_data(data)

class _ClientResponse(aiohttp.client_reqrep.ClientResponse):
    # workaround bug in FlowControlStreamReader
    flow_control_class = _FlowControlStreamReader

the same error
RuntimeError: Cannot pause_reading() when closing

with this code:
import asyncio
import aiohttp

async def crawler(loop):
    url = 'http://www.powertoolsdirect.com/hand-tools/laser-levels'
    async with aiohttp.ClientSession(loop=loop) as session:
        async with session.get(url) as response:
            print(response.status)

if __name__ == '__main__':
    loop = asyncio.get_event_loop()
    loop.run_until_complete(crawler(loop))
    loop.close()

environment:
pip3.5 freeze
aiohttp==1.2.0
async-timeout==1.1.0
chardet==2.3.0
yarl==0.8.1

`which python3.5` -V
Python 3.5.2

Confirm this bug on Ubuntu 16.04 with Python 3.6.0b4 and aiohttp 1.2.0
Traceback:
  File "/home/iaroslavr/projects/crawler.py", line 88, in _get_html     html = await response.text()   File "/home/iaroslavr/.virtualenvs/3.6.0b4/lib/python3.6/site-packages/aiohttp/client_reqrep.py", line 747, in text     yield from self.read()   File "/home/iaroslavr/.virtualenvs/3.6.0b4/lib/python3.6/site-packages/aiohttp/client_reqrep.py", line 718, in read     self._content = yield from self.content.read()   File "/home/iaroslavr/.virtualenvs/3.6.0b4/lib/python3.6/site-packages/aiohttp/streams.py", line 517, in wrapper     result = yield from func(self, *args, **kw)   File "/home/iaroslavr/.virtualenvs/3.6.0b4/lib/python3.6/site-packages/aiohttp/streams.py", line 582, in read     return (yield from super().read(n))   File "/home/iaroslavr/.virtualenvs/3.6.0b4/lib/python3.6/site-packages/aiohttp/streams.py", line 264, in read     raise self._exception   File "/home/iaroslavr/.virtualenvs/3.6.0b4/lib/python3.6/site-packages/aiohttp/parsers.py", line 188, in set_parser     next(p)   File "/home/iaroslavr/.virtualenvs/3.6.0b4/lib/python3.6/site-packages/aiohttp/protocol.py", line 305, in __call__     yield from self.parse_chunked_payload(out, buf)   File "/home/iaroslavr/.virtualenvs/3.6.0b4/lib/python3.6/site-packages/aiohttp/protocol.py", line 348, in parse_chunked_payload     out.feed_data(chunk, len(chunk))   File "/home/iaroslavr/.virtualenvs/3.6.0b4/lib/python3.6/site-packages/aiohttp/protocol.py", line 392, in feed_data     self.out.feed_data(chunk, len(chunk))   File "/home/iaroslavr/.virtualenvs/3.6.0b4/lib/python3.6/site-packages/aiohttp/streams.py", line 573, in feed_data     self._stream.transport.pause_reading()   File "/usr/local/lib/python3.6.0b4/lib/python3.6/asyncio/sslproto.py", line 344, in pause_reading     self._ssl_protocol._transport.pause_reading()   File "/usr/local/lib/python3.6.0b4/lib/python3.6/asyncio/selector_events.py", line 702, in pause_reading     raise RuntimeError('Cannot pause_reading() when closing') RuntimeError: Cannot pause_reading() when closing

#1551 should solve this problem. could anyone test it?

@fafhrd91 I've checked your patch on my staging environment and seems that it fixed the problem! Thanks!

@fafhrd91 Yep, looks like error is gone

fixed by #1551


add to Androdx

Hi, could you elaborate what exactly it is you request?

I assume @chengmenghuiAndroid asks if you might update the libraries you use to match with AndroidX.
I came across an error, while I was using the AndroidX library. In API <24 I had all the time SIGSEGV errors. In API >= 24 the error didn't occur.
I was also able to reproduce this error in a little test project. Interestingly if I created the same test project and did not use the AndroidX library, but the Android support library, there was no error, even in API's lower than 24.
Most likely it would suffice if you update your project dependency of renderscript from android.support.v8.renderscript.RenderScript to androidx.renderscript.

This has maybe something todo with the 64 bit support issue in #16? I will postpne the androidX update because internet dude from stack overflow claims androidX will break the 64 bit support, see: https://stackoverflow.com/a/56477402/774398


Move to API Gateway

Spin up API Gateway architecture in the CDK rather than Lambda URLs

An API Gateway is beneficial because spinning up an API Gateway architecture in the CDK rather than using Lambda URLs directly gives you better control over your API's routing, security, and other features:

Efficient routing and versioning: With API Gateway, you can route different URL paths or request types to different Lambda functions. You can also manage different versions of your API more effectively.
Throttling and scaling: API Gateway provides built-in throttling to protect your back-end services. It also helps manage the scaling of your services.
Security: API Gateway provides several features that allow you to create more secure APIs, such as AWS IAM roles and policies, Amazon Cognito user pools, AWS WAF (a web application firewall), and resource policies.
Monitoring and troubleshooting: API Gateway integrates with Amazon CloudWatch and AWS X-Ray allowing you to monitor API usage and troubleshoot potential issues.

To transfer the domain:
https://repost.aws/knowledge-center/account-transfer-route-53#:~:text=You can transfer a domain,domain by contacting AWS Support.&text=Create a new hosted zone in an AWS account that you own.

Here are the steps to request a new SSL certificate in your new account:
Navigate to AWS Certificate Manager in your AWS Console.
Click on "Request a certificate".
Select "Request a public certificate" and click "Request a certificate".
On the next screen, in the "Add domain names" section, add your domain name. If you plan to have subdomains and you want the certificate to cover them, you can use a wildcard by entering "*.imap-mission.com". If you don't plan to have subdomains, just enter "imap-mission.com".
Click "Next".
On the next page, select "DNS validation".
Click "Review", review your settings, and then click "Confirm and request".
On the validation page, you'll see a CNAME record that you need to add to your Route 53 DNS records for imap-mission.com. This proves to ACM that you have control over the domain.
After you add the CNAME record to Route 53, the Certificate Manager will automatically validate it and issue the SSL certificate, which you can use in your API Gateway setup.
Once the certificate is issued, you'll be able to select it in API Gateway custom domain setup in the same account.
Remember, SSL certificates are region-specific, so make sure to request the certificate in the same region where you're setting up your API Gateway.
Finally, it's worth noting that you cannot transfer SSL certificates between AWS accounts; they're tied to the account where they were issued.

Note: I will make updates according to the results of ruff and black next week. Happy 4th!

This one is definitely done!


feat: Combine multiple examples to one

Closes #139
Summary of Changes
Multiple example docstrings are now combined to one big example docstrings will now each have an example block in Safe-DS docstrings. Previously only the first docstring was saved and others where ignored.

@Masara Needs conflict resolution.


Controller issues

Something seems to be wrong with our gamepad controller. When I first too it out of the box and started charging it, it turned amber which seemed fine. However after some time I disconnected it and it stayed amber. Even pressing the pairing key combo, nothing changed. Only the reset button on the back helped.
Now connecting often fails, as does driving the robot. We are still investigating if this is an issue with this particular controller or with the raspi bluetooth module.
It could well be that the controller is just not charged. When I connected to USB for charging today evening, it did not light up at all. Different cables/different USB outlets did not affect the behavior.

Thanks! I'll try that tomorrow.

Quite frequently, the controller does not pair with the robot. Even when holding it right next to the robot.

Same with my controller. It is not showing any signs of life.

I've encountered difficulties too:
I couldn't connect after I've paired successfully. The PI stated it was connected but the controller was blinking still).
I just tried trust and then connect without the pair, which worked for me.

Closing as stale.

I have the same issue. I can get it to connect and pair again using what is described above but I cannot get it to auto-connect any more by just pushing the home button. Every time it disconnects again, I have to remove to reconnect it using sudo bluetoothctl. Is there a way that I can get it to connect by pushing the home button again?


Crash when saving tree with duplicate View ids

I don't know if this is a worthwhile change for Flow, but I figured I'd post this simple trick here, as well. It prevents users from having subtle bugs from having duplicate View ids in the tree they are saving.

This is a great idea, but I'm not 100% sure that this is in bounds for Flow, or that it wouldn't create headaches for people trying to reuse layout snippets who don't care about the save/restore behavior for those views.
Am I nuts?

I would think copy-pasting layout snippets in the same tree would cause headaches, anyway, if only with findViewById().
But, yeah, I do agree that it's not really Flow's job. 👍


Update command to install salt

The script now contains bash-only instructions.

This change is

@bors-servo r+

:pushpin: Commit 6f7745c has been approved by asajeffrey

:hourglass: Testing commit 6f7745cf021062f416f27aa0688d39bd91c80128 with merge 515156e00ef34fcf8b72616c1ebc7f98f68d2ff3...

:sunny: Test successful - status-travis
Approved by: asajeffrey
Pushing 515156e00ef34fcf8b72616c1ebc7f98f68d2ff3 to master...


Detect a line rather than a word

Which field to tweak so that a word box can be converted to line

I want to know how you mange to recover the shrunk part which was shrunk by EAST before training,
For long words which was mention in the issue that due to small receptive field it cannot cover long length words,
how to get single digits or character detection
and how to get line boundary from words.


Use uber-go/atomic in pkg/autodiscovery

This hasRunOnce functionality is destined for the scrap-heap, but until
then we can still use a properly-aligned atomic.
What does this PR do?
Uses github.com/uber-go/atomic for the atomic used to track whether AD hasRunOnce.
Motivation
See #11716.
Additional Notes

Possible Drawbacks / Trade-offs

Describe how to test/QA your changes
Run the agent in a container environment with logs_config.container_collect_all enabled, and verify that it logs a container without any annotations or labels.
Reviewer's Checklist

[ ] If known, an appropriate milestone has been selected; otherwise the Triage milestone is set.
[ ] Use the major_change label if your change either has a major impact on the code base, is impacting multiple teams or is changing important well-established internals of the Agent. This label will be use during QA to make sure each team pay extra attention to the changed behavior. For any customer facing change use a releasenote.
[ ] A release note has been added or the changelog/no-changelog label has been applied.
[ ] Changed code has automated tests for its functionality.
[ ] Adequate QA/testing plan information is provided if the qa/skip-qa label is not applied.
[ ] At least one team/.. label has been applied, indicating the team(s) that should QA this change.
[ ] If applicable, docs team has been notified or an issue has been opened on the documentation repo.
[ ] If applicable, the need-change/operator and need-change/helm labels have been applied.
[ ] If applicable, the config template has been updated.

It looks like the macos failures are unrelated:
Error: Fetching /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core failed!
To restore the stashed changes to /usr/local/Homebrew, run:
  cd /usr/local/Homebrew && git stash pop


 Appears to be an Arduino HTTP lib.

Thanks!


ambiguous identifier: 'QueryParams'

when i just do:
import mummy

i get the error:
Hint: used config file '/home/david/.choosenim/toolchains/nim-2.0.2/config/nim.cfg' [Conf]
Hint: used config file '/home/david/.choosenim/toolchains/nim-2.0.2/config/config.nims' [Conf]
.................................................................................................................................................................................
/home/david/.nimble/pkgs2/mummy-0.4.1-4b42e2d861877701288ea069a743dc74b6786e02/mummy.nim(43, 19) Error: ambiguous identifier: 'QueryParams' -- use one of the following:
  queryparams.QueryParams: QueryParams
  queryparams.QueryParams: QueryParams

shell returned 1

did i messed up my nimble installation?

This is something nimble related or the Github Actions tests would not pass.
Nim 2.0 unfortunately pretty much broke nimble so I would imagine it could be pkgs/ vs pkgs2/ and installed with Nim 1.x vs 2.x conflicts. A clean reset of the nimble installed stuff would be the real test which is what the Github Actions attempt to ensure works.
Re-runniing now: https://github.com/guzba/mummy/actions/runs/7824996753

Ok it was indeed some nimble stuff (i think it was the nimble cache), i did:
cd ~/.nimble/pkgs
rm -r zippy-0.10.*
rm webby*
rm -r webby*
rm -r crunchy*
rm -r nimsimd*
cd ~/.nimble/pkgs2
rm -r mummy*
rm -r zippy*
rm -r webby*
rm -r crunchy*
rm -r nimsimd*
rm -r mummy*
nimble install mummy
cd ~/.cache/nim    # <--- this might have fixed it
rm -r *

should have done this before, so sorry for the noise.


Feat socket io

This is not quite ready for review yet.
End-user friendly description of the problem this fixes or functionality that this introduces

[ ] Include this change in the Release Notes. If checked, you must provide an end-user friendly description for your change below

Give a summary of what the PR does, explaining any non-trivial design decisions

Link of any specific issues this addresses

Will this make https://github.com/All-Hands-AI/OpenHands/issues/5019 easier to implement by decoupling the Session from the websocket?

Will this make https://github.com/All-Hands-AI/OpenHands/issues/5019 easier to implement by decoupling the Session from the websocket?

I don't think so. We'll have to think carefully about that one--the agent loop will continue running in perpetuity if we don't explicitly close it on some event

the agent loop will continue running in perpetuity if we don't explicitly close it on some event
I think it will be closed by a timeout event, with a configurable idle time of say N minutes (in our setup, we would close out the session after many hours)

the agent loop will continue running in perpetuity if we don't explicitly close it on some event I think it will be closed by a timeout event, with a configurable idle time of say N minutes (in our setup, we would close out the session after many hours)
Heck, when the EventStreamRuntime is instantiated, it can just inject an async event after sleeping N minutes to terminate itself into the event loop.
E.g.
asyncio.sleep(...)
self.close()

I'll update the PR to make this value configurable - so that there is a delay before the runtime is closed out.


Don't accept random arguments for skip/unskip

Currently the bot will accept skip and unskip with either a random second argument, or skip tomorrow followed by more arguments.

Great catch! tyty @parsn1psoup !


Add feature for menu sharing

Addresses #51
Uses screenshot api to take a long picture of the menu tableview

LGTM.  Let me know when you address all the comments.  Should be ready to merge after that

Eric reviewed in person

Merged 4eafdb4c443fcf30c805ddc3ffaa6c7c4a11f52d


Search(ctrl+shift+F) doesn't work for C++

1.Open a c++ project folder in vscode and no files opened;
2.Click : Search(ctrl+shift+F) tab on the left;
3.Enter a class name which exsit in c++ project folder to search edit box;
4.The search result is empty; And I think the file in the folder is not searched;
5. But if open the files of the opened folder, it will search all the opened files, and the result will shown;

/needsMoreInfo
Are these classes in excluded/ignored folders?
See https://code.visualstudio.com/docs/editor/codebasics#_advanced-search-options especially paragraph with
"Also note the Use Exclude Settings and Ignore Files toggle button in the files to exclude box."
Files in open editors use different mechanism for search and will be included even if outside of workspace root or in excluded files.


Github action 22.3.0 does not work on Linux anymore

When using:
     - uses: graalvm/setup-graalvm@v1
        with:
          version: 'latest'
          java-version: '17'
          components: 'js'

We got:
Run graalvm/setup-graalvm@v1
/usr/bin/tar xz --warning=no-unknown-keyword --overwrite -C /home/runner/work/_temp/2001f429-3c1a-4eab-a464-d104551b4859 -f /home/runner/work/_temp/9b690481-8975-4f3b-90af-030c430177c1
Adding graalvm-ce-java17-linux 22.3.0 to tool-cache ...
/opt/hostedtoolcache/graalvm-ce-java17-linux/22.3.0/x64/graalvm-ce-java17-22.3.0/bin/gu --non-interactive install --no-progress js
Downloading: Component catalog from www.graalvm.org
Could not load channel https://www.graalvm.org/component-catalog/v2/graal-updater-component-catalog-java17.properties: Unsupported GraalVM version: 22.3.0, platform linux/amd64
Error: Unsupported GraalVM version: 22.3.0, platform linux/amd64
Error: The process '/opt/hostedtoolcache/graalvm-ce-java17-linux/22.3.0/x64/graalvm-ce-java17-22.3.0/bin/gu' failed with exit code 3``

We've just rolled out the 22.3 release and your build ran during the small window whether some of our bits were inconsistent. This should now work again. I'm going to follow up with our infra team to find a way how we can roll this out without breaking builds in the future.

Hello Everyone
Today we have started getting the similar issue for version 23.0.3. We were seeing sometime back as below:
Error: Unsupported GraalVM version: 23.0.3, platform linux/amd64

Although in the latest runs the version part is gone in the error:
Error: Unsupported GraalVM release: OS: linux, arch: amd64, java version: 17

Hey @mdubey-tw, thanks for reporting! We are rolling out new patch releases and there are unfortunately some technical issues that we are currently working on. Apologies for the inconvenience this is causing. I expect this to be resolve in a couple of hours.

Sure @fniephaus!
Thank you for your quick response!


Bootstrap setting up cross signing JWT

Checklist

[X] I could not find a solution in the documentation, the existing issues or discussions.
[ ] I already asked for help in the chat

In which Project did the bug appear?
Famedly App
If you selected "Other" as Project, please enter in which project the bug occurred.
No response
On which platform did the bug appear?
Android, iOS
Matrix API Version
172
Describe the problem caused by this bug
Hello,
im using dart sdk with JWT token from another auth service and when doing bootstrap i get to this point:
flutter: [Matrix] BootstrapState: BootstrapState.loading
flutter: [Matrix] Upload device signing keys.

after that i get flutter: [Matrix] [Bootstrapping] Error setting up cross signing - M_FORBIDDEN: Require additional authentication
When trying with username password, it works normally.
Dont know if its issue of this SDK so just to be sure i created issue on synapse repo #16508
Steps To Reproduce

use jwt authentication
try to do uploadCrossSigningKeys https://pub.dev/documentation/matrix_api_lite/latest/matrix_api_lite/MatrixApi/uploadCrossSigningKeys.html
see error

Screenshots or Logs
jwt_config:
enabled: true
secret: |-
-----BEGIN CERTIFICATE-----
REDACTED
-----END CERTIFICATE-----
algorithm: "RS256"
subject_claim: "sub"
issuer: "REDACTED"
audiences:
- "REDACTED"
2023-10-17 03:04:10,463 - synapse.access.http.8008 - 465 - INFO - POST-16 - <IP_ADDRESS> - 8008 - {@REDACTED:REDACTED.com} Processed request: 0.019sec/0.000sec (0.006sec, 0.001sec) (0.002sec/0.012sec/6) 61B 401 "POST /_matrix/client/v3/keys/device_signing/upload HTTP/1.1" "Dart/3.1 (dart:io)" [0 dbevts]

Security related
No response

Hey, you just need to implement handling this error and perform the UIA steps. Please read the matrix specification for more info.
Closing as it is not a bug


⚠ gorzdrav.org has degraded performance

In 9d094e9, gorzdrav.org (https://gorzdrav.org) experienced degraded performance:

HTTP code: 200
Response time: 9170 ms

Resolved: gorzdrav.org performance has improved in b0ec9c4.


jackett: 0.14.365 -> 0.16.175, add aarch64 support

Motivation for this change
The existing jackett derivation installed the x86_64 binary on all Linux platforms, and was thus broken on aarch64, e.g. Raspberry Pi. The upstream project provides ARM64 Linux binaries: this change adds support for those, and also bumps the packaged version to the latest upstream release.
Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[x] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

@GrahamcOfBorg build jackett

Thanks @offlinehacker!

Can #81841, #83980, and this get backported to 20.03?


arm64 image tagged as amd64

When trying to build for amd64 I get the error below because the image is actually arm64/V8. I was able to build the amd64 image locally.
$ docker run --platform linux/amd64 --rm -v "${PWD}":/usr/src/zk -w /usr/src/zk mickaelmenu/zk-xcompile:linux-amd64 /bin/bash -c 'make'
Unable to find image 'mickaelmenu/zk-xcompile:linux-amd64' locally
linux-amd64: Pulling from mickaelmenu/zk-xcompile
Digest: sha256:5888bd7ecc5961f4a636c2697f4313ca9f4df3442baa1b0c99b35ede5402853c
Status: Image is up to date for mickaelmenu/zk-xcompile:linux-amd64
docker: Error response from daemon: image with reference mickaelmenu/zk-xcompile:linux-amd64 was found but does not match the specified platform: wanted linux/amd64, actual: linux/arm64/v8.
See 'docker run --help'.

I'm not sure how to fix this, as I'm no Docker expert. Any suggestion?

The command in the README looks ok, the -t mickaelmenu/zk-xcompile:linux-amd64 option tells docker to label the image with the name:tag in the argument. I'm guessing that the last time the amd64 image was built and uploaded to the registry the command got messed up and it was incorrectly tagged as arm64 instead.
Repeating the build & push steps with the correct tag should fix it:
docker build -t mickaelmenu/zk-xcompile:linux-amd64 ./linux-amd64
docker push mickaelmenu/zk-xcompile:linux-amd64

I just did, let me know if it fixes the issue for you.

I see that it is still marked as linux/arm64/v8 🤷♂
https://hub.docker.com/layers/mickaelmenu/zk-xcompile/linux-amd64/images/sha256-6d727ac5309cf98f7b9ecbf2082ac3a2b8dd0b85e7d86833946460ed35954e6f?context=repo

I see! I don't usually build different images for different platforms, so I didn't think of this issue. Docker will build for the platform defined in the env variable DOCKER_DEFAULT_PLATFORM, regardless of the os/arch labels in the Dockerfile. To change it for each build, you can either change the environment variable before running docker build, or add the --platform flag to the FROM line in each Dockerfile, e.g.:
FROM --platform linux/amd64 golang:1.18

Thank you, it looks like it worked and now I'm able to compile the binaries for Linux! Thanks a lot for your help.


fix incorrect use log and optimized log output

Please provide a description of this PR:
fix incorrect use log ：pilot/pkg/xds/discovery.go
optimized log output：other

@nicole-lihui: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

release-notes_istio
918b27cd32b5f111cb59b372423fc2156999031d
link
true
/test release-notes_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

/test release-notes_istio


How to specify the datastore name when creating or updating a data asset using Azure Machine Learning Python SDK V2

It is unclear how to provide the datastore name when creating or updating a data asset. The following example from documentation does not specify the data store name.
my_data = Data(
    name="credit-card",
    version=v1,
    description="Credit card data",
    path=my_path,
    type=AssetTypes.URI_FILE,
)

## create data asset if it doesn't already exist:
try:
    data_asset = ml_client.data.get(name="credit-card", version=v1)
    print(
        f"Data asset already exists. Name: {my_data.name}, version: {my_data.version}"
    )
except:
    ml_client.data.create_or_update(my_data)
    print(f"Data asset created. Name: {my_data.name}, version: {my_data.version}")

I think the Data class has an attribute named datastore for this purpose, but I cannot location any documentation regarding this. Also, it would be great to document the attribute for specifying the location inside datastore where the data will be uploaded.

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: be5f72f6-bc45-1717-3d02-b0d53cf71c30
Version Independent ID: 8e626e10-e162-d78b-97b1-4c1cefca4808
Content: Create Data Assets - Azure Machine Learning
Content Source: articles/machine-learning/how-to-create-data-assets.md
Service: machine-learning
Sub-service: mldata
GitHub Login: @SturgeonMi
Microsoft Alias: xunwan

@sutharzan-ch
Thanks for your feedback! We will investigate and update as appropriate.

@SturgeonMi
Please review it.

@sutharzan-ch
Thank you for bringing this to our attention.
I've delegated this to content author @SturgeonMi, who will review it and offer their insightful opinions.

@SturgeonMi
Could you please review add comments on this, update as appropriate.


Scenario Arguments

This task sets up the ability to pass arguments into the scenario for it to process separately. This can allow the scenario to activate different models, perform different scenarios, and modify inputs based on the arguments given.

This is implemented for the most part in the Model Reference Architecture branch.


how to check if notification is in background or in foreground

if you pass content_available flag true, you will receive one when app is in background and another one when user resume the app.

How can I check if the FCM.on('notification', (notif) => {}) was called on background or foreground?

also how can I check if the user has clicked on my notification?

On react-native-push-notification - https://github.com/zo0r/react-native-push-notification#handling-notifications
there is a field that show if it was received on foreground or not
foreground: false, // BOOLEAN: If the notification was received in foreground or not

I've tried to use AppState.currentState but it is showing in background

do you mean when you receive the application when app is in foreground, AppState.currentState returns background?

This is my scenario:
I close my app (send it to background)
I send a push notification with content-available: true, then I receive a notification on FCM.on('notification', (notif) => {}), in this case the AppState.currentState returns background
When I click the notification, it will receive another notification on FCM.on, and the AppState.currentState is background yet, so I don't know if the user clicked or not in my notification. the AppState changes after the FCM.on
I think that why react-native-push-notification (https://github.com/zo0r/react-native-push-notification#handling-notifications) has two flags to help with this scenario:
foreground: false, // BOOLEAN: If the notification was received in foreground or not
userInteraction: false, // BOOLEAN: If the notification was opened by the user from the notification area or not

what do think do use a setTimeout to check wait a little bit to appState to go active to check if the notification was clicked or not?

@sibelius timeout won't help because the delay is happening in native world.

how r u handling this case for now?

I don't have a use case for now. But I will investigate how to do it the week.

this kind of works for now
the first AppState.currentState it is in background, the one inside setTimeout is active
this.notificationUnsubscribe = FCM.on('notification', (notif) => {
      console.log('APPSTATE: ', AppState.currentState);
      setTimeout(() => {
        console.log('APPSTATE TIME: ', AppState.currentState);
      }, 100)
})

My use case is send the user to a specific screen of my app when it click in the notification (deeplinking)

i guess it does take some time for app state for be updated. perhaps this check should happen inside module rather than relying on the javascript propery

are you willing to review and accept a PR that makes this check inside native module?
similar to
foreground: false, // BOOLEAN: If the notification was received in foreground or not
userInteraction: false, // BOOLEAN: If the notification was opened by the user from the notification area or not

or opened_from_tray https://github.com/evollu/react-native-fcm/pull/87 ?

i think it will have the same issue as you have as it is just checking in javascript world

is opened_from_tray enough to solve your issue?

when I click a notification on ios 10, it does not call FCM.on('notification', (notif) => {})

hmm.. let me try ios 10

FCM.presentLocalNotification is not working on ios 10 either

have you tried ios 10 only or it works on ios 9?

FCM.on('notification', (notif) => {}) is working fine on ios 9
I haven't tested presentLocalNotification on ios 9

@sibelius @evollu I can't run on IOS 10 either.

downloading ios 10

@evollu I can confirm that my app that is currently running the previous version works normally on IOS 9 and 10

@rturk you mean on('otification')?

@sibelius when you test local notification, is the app in foreground?

it was in foreground

you can't won't see any notification when you are in foreground iOS. try sent a timeout and go background or use schedulelocalnotification

it will only show if the app is in background?

yes. that is how remote notification work too. you won't see the banner

so I can only use localNotification when the app is in background?
could u add this info on README.md as well?
why it was working before?

no, you can only see notification when app is in background.
what is your use case? do you want to show a banner when app is in foreground? An alert makes more sense to me in this scenario.

@sibelius @rturk just tested in iOS 10, it is working for me (remote notification). What are your cases?

For the local notification an alert makes more sense for sure!
If the app is in background, I wanna go to a specific screen when the user clicks the notification.
My problem is that FCM.on('notification') is not being called anymore when the user clicks in the notification

is that notification local or remote? local notification triggers FCM.on('localNotification')

it is a remote notification

wired, mine gets triggered correctly

r u in debugging mode?

AppState can tell you if the app is in the foreground or background, and notify you when the state changes.
https://facebook.github.io/react-native/docs/appstate.html

fixed on master
tks for all the help @evollu


feat: Add new transfers table

Description
This pull request creates the transfers table. Addresses issue #18.
Tests

[x] Please provide evidence of your successful dbt run / dbt test here

dbt run
root@079c68a4191b:/terra/models/silver# dbt run --select silver__transfers
17:43:09  Running with dbt=1.2.0
17:43:14  Warning: the `type_timestamp` macro is now provided in dbt Core. It is no longer available in dbt_utils and backwards compatibility will be removed in a future version of the package. Use `type_timestamp` (no prefix) instead. The terra.dbt_expectations_expect_row_values_to_have_recent_data_core__ez_staking_BLOCK_TIMESTAMP__day__1 model triggered this warning.
17:43:14  Warning: the `type_timestamp` macro is now provided in dbt Core. It is no longer available in dbt_utils and backwards compatibility will be removed in a future version of the package. Use `type_timestamp` (no prefix) instead. The terra.dbt_expectations_expect_row_values_to_have_recent_data_core__ez_staking_BLOCK_TIMESTAMP__day__1 model triggered this warning.
17:43:14  Warning: the `type_timestamp` macro is now provided in dbt Core. It is no longer available in dbt_utils and backwards compatibility will be removed in a future version of the package. Use `type_timestamp` (no prefix) instead. The terra.dbt_expectations_expect_row_values_to_have_recent_data_core__ez_staking_BLOCK_TIMESTAMP__day__1 model triggered this warning.
17:43:14  Warning: the `type_timestamp` macro is now provided in dbt Core. It is no longer available in dbt_utils and backwards compatibility will be removed in a future version of the package. Use `type_timestamp` (no prefix) instead. The terra.dbt_expectations_expect_row_values_to_have_recent_data_core__ez_staking_BLOCK_TIMESTAMP__day__1 model triggered this warning.
17:43:14  Warning: the `type_timestamp` macro is now provided in dbt Core. It is no longer available in dbt_utils and backwards compatibility will be removed in a future version of the package. Use `type_timestamp` (no prefix) instead. The terra.dbt_expectations_expect_row_values_to_have_recent_data_core__ez_staking_BLOCK_TIMESTAMP__day__1 model triggered this warning.
17:43:14  Warning: the `type_timestamp` macro is now provided in dbt Core. It is no longer available in dbt_utils and backwards compatibility will be removed in a future version of the package. Use `type_timestamp` (no prefix) instead. The terra.dbt_expectations_expect_row_values_to_have_recent_data_core__ez_staking_BLOCK_TIMESTAMP__day__1 model triggered this warning.
17:43:14  Warning: the `type_timestamp` macro is now provided in dbt Core. It is no longer available in dbt_utils and backwards compatibility will be removed in a future version of the package. Use `type_timestamp` (no prefix) instead. The terra.dbt_expectations_expect_row_values_to_have_recent_data_core__ez_staking_BLOCK_TIMESTAMP__day__1 model triggered this warning.
17:43:14  Warning: the `dateadd` macro is now provided in dbt Core. It is no longer available in dbt_utils and backwards compatibility will be removed in a future version of the package. Use `dateadd`
(no prefix) instead. The terra.dbt_expectations_expect_row_values_to_have_recent_data_core__ez_staking_BLOCK_TIMESTAMP__day__1 model triggered this warning.
17:43:14  Warning: the `type_timestamp` macro is now provided in dbt Core. It is no longer available in dbt_utils and backwards compatibility will be removed in a future version of the package. Use `type_timestamp` (no prefix) instead. The terra.dbt_expectations_expect_row_values_to_have_recent_data_core__ez_staking_BLOCK_TIMESTAMP__day__1 model triggered this warning.
17:43:14  Warning: the `type_timestamp` macro is now provided in dbt Core. It is no longer available in dbt_utils and backwards compatibility will be removed in a future version of the package. Use `type_timestamp` (no prefix) instead. The terra.dbt_expectations_expect_row_values_to_have_recent_data_silver__staking_BLOCK_TIMESTAMP__day__1 model triggered this warning.
17:43:14  Warning: the `type_timestamp` macro is now provided in dbt Core. It is no longer available in dbt_utils and backwards compatibility will be removed in a future version of the package. Use `type_timestamp` (no prefix) instead. The terra.dbt_expectations_expect_row_values_to_have_recent_data_silver__staking_BLOCK_TIMESTAMP__day__1 model triggered this warning.
17:43:14  Warning: the `type_timestamp` macro is now provided in dbt Core. It is no longer available in dbt_utils and backwards compatibility will be removed in a future version of the package. Use `type_timestamp` (no prefix) instead. The terra.dbt_expectations_expect_row_values_to_have_recent_data_silver__staking_BLOCK_TIMESTAMP__day__1 model triggered this warning.
17:43:14  Warning: the `type_timestamp` macro is now provided in dbt Core. It is no longer available in dbt_utils and backwards compatibility will be removed in a future version of the package. Use `type_timestamp` (no prefix) instead. The terra.dbt_expectations_expect_row_values_to_have_recent_data_silver__staking_BLOCK_TIMESTAMP__day__1 model triggered this warning.
17:43:14  Warning: the `type_timestamp` macro is now provided in dbt Core. It is no longer available in dbt_utils and backwards compatibility will be removed in a future version of the package. Use `type_timestamp` (no prefix) instead. The terra.dbt_expectations_expect_row_values_to_have_recent_data_silver__staking_BLOCK_TIMESTAMP__day__1 model triggered this warning.
17:43:14  Warning: the `type_timestamp` macro is now provided in dbt Core. It is no longer available in dbt_utils and backwards compatibility will be removed in a future version of the package. Use `type_timestamp` (no prefix) instead. The terra.dbt_expectations_expect_row_values_to_have_recent_data_silver__staking_BLOCK_TIMESTAMP__day__1 model triggered this warning.
17:43:14  Warning: the `type_timestamp` macro is now provided in dbt Core. It is no longer available in dbt_utils and backwards compatibility will be removed in a future version of the package. Use `type_timestamp` (no prefix) instead. The terra.dbt_expectations_expect_row_values_to_have_recent_data_silver__staking_BLOCK_TIMESTAMP__day__1 model triggered this warning.
17:43:14  Warning: the `dateadd` macro is now provided in dbt Core. It is no longer available in dbt_utils and backwards compatibility will be removed in a future version of the package. Use `dateadd`
(no prefix) instead. The terra.dbt_expectations_expect_row_values_to_have_recent_data_silver__staking_BLOCK_TIMESTAMP__day__1 model triggered this warning.
17:43:14  Warning: the `type_timestamp` macro is now provided in dbt Core. It is no longer available in dbt_utils and backwards compatibility will be removed in a future version of the package. Use `type_timestamp` (no prefix) instead. The terra.dbt_expectations_expect_row_values_to_have_recent_data_silver__staking_BLOCK_TIMESTAMP__day__1 model triggered this warning.
17:43:15  Found 50 models, 198 tests, 0 snapshots, 0 analyses, 684 macros, 2 operations, 0 seed files, 44 sources, 0 exposures, 0 metrics
17:43:15
17:43:33
17:43:33  Running 2 on-run-start hooks
17:43:33  1 of 2 START hook: terra.on-run-start.0 ........................................ [RUN]
17:43:33  1 of 2 OK hook: terra.on-run-start.0 ........................................... [OK in 0.00s]
17:43:33  2 of 2 START hook: terra.on-run-start.1 ........................................ [RUN]
17:43:36  2 of 2 OK hook: terra.on-run-start.1 ........................................... [SUCCESS 1 in 2.05s]
17:43:36
17:43:36  Concurrency: 4 threads (target='dev')
17:43:36
17:43:36  1 of 1 START incremental model silver.transfers ................................ [RUN]
17:43:55  1 of 1 OK created incremental model silver.transfers ........................... [SUCCESS 1 in 18.80s]
17:43:55
17:43:55  Finished running 1 incremental model, 2 hooks in 0 hours 0 minutes and 40.38 seconds (40.38s).
17:43:55
17:43:55  Completed successfully
17:43:55
17:43:55  Done. PASS=1 WARN=0 ERROR=0 SKIP=0 TOTAL=1

dbt test
root@079c68a4191b:/terra/models/silver# dbt test --select silver__transfers
17:44:09  Running with dbt=1.2.0
17:44:14  Found 50 models, 198 tests, 0 snapshots, 0 analyses, 684 macros, 2 operations, 0 seed files, 44 sources, 0 exposures, 0 metrics
17:44:14
17:44:32
17:44:32  Running 2 on-run-start hooks
17:44:32  1 of 2 START hook: terra.on-run-start.0 ........................................ [RUN]
17:44:32  1 of 2 OK hook: terra.on-run-start.0 ........................................... [OK in 0.00s]
17:44:32  2 of 2 START hook: terra.on-run-start.1 ........................................ [RUN]
17:44:34  2 of 2 OK hook: terra.on-run-start.1 ........................................... [SUCCESS 1 in 1.93s]
17:44:34
17:44:35  Concurrency: 4 threads (target='dev')
17:44:35
17:44:35  1 of 18 START test dbt_expectations_expect_column_values_to_be_in_type_list_silver__transfers_BLOCK_ID__NUMBER__FLOAT  [RUN]
17:44:35  2 of 18 START test dbt_expectations_expect_column_values_to_be_in_type_list_silver__transfers_BLOCK_TIMESTAMP__TIMESTAMP_NTZ  [RUN]
17:44:35  3 of 18 START test dbt_expectations_expect_column_values_to_be_in_type_list_silver__transfers_CHAIN_ID__VARCHAR  [RUN]
17:44:35  4 of 18 START test dbt_expectations_expect_column_values_to_be_in_type_list_silver__transfers_MESSAGE_INDEX__NUMBER  [RUN]
17:44:39  1 of 18 PASS dbt_expectations_expect_column_values_to_be_in_type_list_silver__transfers_BLOCK_ID__NUMBER__FLOAT  [PASS in 4.34s]
17:44:39  5 of 18 START test dbt_expectations_expect_column_values_to_be_in_type_list_silver__transfers_MESSAGE_TYPE__VARCHAR  [RUN]
17:44:39  4 of 18 PASS dbt_expectations_expect_column_values_to_be_in_type_list_silver__transfers_MESSAGE_INDEX__NUMBER  [PASS in 4.39s]
17:44:39  6 of 18 START test dbt_expectations_expect_column_values_to_be_in_type_list_silver__transfers_TRANSFER_ID__STRING__VARCHAR  [RUN]
17:44:39  2 of 18 PASS dbt_expectations_expect_column_values_to_be_in_type_list_silver__transfers_BLOCK_TIMESTAMP__TIMESTAMP_NTZ  [PASS in 4.41s]
17:44:39  7 of 18 START test dbt_expectations_expect_column_values_to_be_in_type_list_silver__transfers_TX_ID__STRING__VARCHAR  [RUN]
17:44:39  3 of 18 PASS dbt_expectations_expect_column_values_to_be_in_type_list_silver__transfers_CHAIN_ID__VARCHAR  [PASS in 4.53s]
17:44:39  8 of 18 START test dbt_expectations_expect_column_values_to_be_in_type_list_silver__transfers_TX_SUCCEEDED__BOOLEAN  [RUN]
17:44:43  5 of 18 PASS dbt_expectations_expect_column_values_to_be_in_type_list_silver__transfers_MESSAGE_TYPE__VARCHAR  [PASS in 3.88s]
17:44:43  9 of 18 START test dbt_utils_unique_combination_of_columns_silver__transfers_transfer_id  [RUN]
17:44:44  6 of 18 PASS dbt_expectations_expect_column_values_to_be_in_type_list_silver__transfers_TRANSFER_ID__STRING__VARCHAR  [PASS in 4.36s]
17:44:44  10 of 18 START test not_null_silver__transfers_BLOCK_ID ........................ [RUN]
17:44:44  8 of 18 PASS dbt_expectations_expect_column_values_to_be_in_type_list_silver__transfers_TX_SUCCEEDED__BOOLEAN  [PASS in 4.31s]
17:44:44  11 of 18 START test not_null_silver__transfers_BLOCK_TIMESTAMP ................. [RUN]
17:44:44  7 of 18 PASS dbt_expectations_expect_column_values_to_be_in_type_list_silver__transfers_TX_ID__STRING__VARCHAR  [PASS in 4.65s]
17:44:44  12 of 18 START test not_null_silver__transfers_CHAIN_ID ........................ [RUN]
17:44:47  10 of 18 PASS not_null_silver__transfers_BLOCK_ID .............................. [PASS in 3.65s]
17:44:47  13 of 18 START test not_null_silver__transfers_MESSAGE_INDEX ................... [RUN]
17:44:47  11 of 18 PASS not_null_silver__transfers_BLOCK_TIMESTAMP ....................... [PASS in 3.57s]
17:44:47  14 of 18 START test not_null_silver__transfers_MESSAGE_TYPE .................... [RUN]
17:44:48  12 of 18 PASS not_null_silver__transfers_CHAIN_ID .............................. [PASS in 3.62s]
17:44:48  15 of 18 START test not_null_silver__transfers_TRANSFER_ID ..................... [RUN]
17:44:48  9 of 18 PASS dbt_utils_unique_combination_of_columns_silver__transfers_transfer_id  [PASS in 5.11s]
17:44:49  16 of 18 START test not_null_silver__transfers_TX_ID ........................... [RUN]
17:44:51  14 of 18 PASS not_null_silver__transfers_MESSAGE_TYPE .......................... [PASS in 3.54s]
17:44:51  17 of 18 START test not_null_silver__transfers_TX_SUCCEEDED .................... [RUN]
17:44:51  13 of 18 PASS not_null_silver__transfers_MESSAGE_INDEX ......................... [PASS in 3.66s]
17:44:51  18 of 18 START test unique_silver__transfers_TRANSFER_ID ....................... [RUN]
17:44:51  15 of 18 PASS not_null_silver__transfers_TRANSFER_ID ........................... [PASS in 3.65s]
17:44:52  16 of 18 PASS not_null_silver__transfers_TX_ID ................................. [PASS in 3.46s]
17:44:55  17 of 18 PASS not_null_silver__transfers_TX_SUCCEEDED .......................... [PASS in 3.65s]
17:44:55  18 of 18 PASS unique_silver__transfers_TRANSFER_ID ............................. [PASS in 3.72s]
17:44:55
17:44:55  Finished running 18 tests, 2 hooks in 0 hours 0 minutes and 40.90 seconds (40.90s).
17:44:55
17:44:55  Completed successfully
17:44:55
17:44:55  Done. PASS=18 WARN=0 ERROR=0 SKIP=0 TOTAL=18

[x] Any comparison between prod and dev for any schema change

Checklist

[x] Follow dbt style guide
[x] Tag the person(s) responsible for reviewing proposed changes
[x] Notes to deployment, if a full-refresh is needed for any table
No need
[x] Run git merge main to pull any changes from remote into your branch prior to merge.

All tests have passed. Approved


Precompute function and rebuild the AST sqlstatement

Usually, we hope that the elements in the SQL statement will be parsed and processed before they are actually submitted to the executor (such as checking the permissions of fields and the number of partitions, etc.), but some functions, including some UDFs defined by users, make it difficult to preprocess these elements. For example, the commonly used DATEADD function, if users need to check the partition range involved in the SQL query before submitting it to the executor, it will be quite difficult. In fastsql, we provide a good Function function to define functions and use SQLEvalVisitorUtil to calculate the results of functions and write them into functions, but we do not reconstruct the original SqlStatement (using the results of functions to replace the function expression). Here, I have made a complement to this.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.lizehan seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.lizehan seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.

git push提交使用的user.name和user.email和github账户不一致，签不了CLAassistant，关了这个PR重新提交吧，提交前查看配置的账户和邮箱，要与github账号一致。
$ git config --global -l


Force and assume SSL

Context
This will ensure the app always uses secure cookies and Strict-Transport-Security headers but allow for non-secure requests at the application layer as we encrypt traffic at our ingress layer.
Changes proposed in this pull request

Set config.force_ssl = true and config.assume_ssl = true in config/environments/production.rb.

Guidance to review

Visit the review app.
Check the headers.
The cookie should have secure set on it.
The Strict-Transport-Security header should be set.

Trello Card

https://trello.com/c/FJUKdouv/417-when-the-session-cookie-is-set-it-should-be-marked-secure
https://trello.com/c/gTL5sMZj/418-setup-our-security-headers

This forces cookies to be secure ✅
But it change the Strict-Transport-Security or Content-Security-Policy headers. Is it supposed to, or will that be addressed in a separate PR?
The main branch (staging env) already has a CSP header:
$ curl -sI "https://staging.claim-funding-for-mentor-training.education.gov.uk" | grep -i "Strict-Transport-Security\|Content-Security-Policy"
strict-transport-security: max-age=15724800; includeSubDomains

Compared to the review app for this PR:
$ curl -sI "https://manage-school-placements-534.test.teacherservices.cloud" | grep -i "Strict-Transport-Security\|Content-Security-Policy"
strict-transport-security: max-age=15724800; includeSubDomains

The two are identical 🤔

https://github.com/DFE-Digital/itt-mentor-services/pull/535 adds the CSP headers.


opentrack-user-interface.so: invalid ELF header

Describe the bug
opentrack crashes with
/nix/store/khm7x0lhf4j8mc43q4lycv1ylb532xfj-opentrack-2.3.13/bin/opentrack: error while loading shared libraries: /nix/store/khm7x0lhf4j8mc43q4lycv1ylb532xfj-opentrack-2.3.13/libexec/opentrack/opentrack-user-interface.so: invalid ELF header

Steps To Reproduce
Steps to reproduce the behavior:

nix run nixpkgs#opentrack

Expected behavior
No crash
Additional context
It appears that opentrack-user-interface.so is wrapped by a Qt hook.
Also affected on 22.05.
Reported on Reddit
Notify maintainers
@zaninime
Metadata

system: "x86_64-linux"
host os: Linux 5.15.59, NixOS, 22.05 (Quokka), 22.05.20220806.72f492e
multi-user?: yes
sandbox: yes
version: nix-env (Nix) 2.8.1
nixpkgs: /etc/nix/channels/nixos

Thanks for reporting, once the above PR is merged I will also submit a backport to 22.05. There was indeed a problem with wrapping (but was working in the past, as I was using it almost daily).


ERROR: tcpserver 41826 : error writing message: Broken pipe

Hi @praveen-palanisamy ,
Will this error be the problem?
ERROR: tcpserver 37004 : error reading message: End of file
ERROR: tcpserver 37003 : error writing message: Broken pipe

I set "discrete_actions": False to find out how well can the model worked in the continuous action area, and received this error.

Hi @Panshark ,
The two error lines in your post indicates that the connection to the CARLA server was broken. Likely because the server stopped or is not running?
Please make sure the server is running and try again. If you run into issues again, please attach the full log file in your reply.

Hi @praveen-palanisamy ,
Here is the full log:
Initializing new Carla server...
Initializing new Carla server...
WARNING: No trained model found for this environment. Training from scratch.
Initializing new Carla server...
Initializing new Carla server...
Initializing new Carla server...
Initializing new Carla server...
Initializing new Carla server...
Initializing new Carla server...
Start pos 21 ([-1.0, 2.0]), end 65 ([0.0, 1.0])
Starting new episode...
Start pos 21 ([-1.0, 2.0]), end 65 ([0.0, 1.0])
Starting new episode...
Start pos 21 ([-1.0, 2.0]), end 65 ([0.0, 1.0])
Starting new episode...
Start pos 21 ([-1.0, 2.0]), end 65 ([0.0, 1.0])
Starting new episode...
Start pos 21 ([-1.0, 2.0]), end 65 ([0.0, 1.0])
Starting new episode...
Start pos 21 ([-1.0, 2.0]), end 65 ([0.0, 1.0])
Starting new episode...
Start pos 21 ([-1.0, 2.0]), end 65 ([0.0, 1.0])
Starting new episode...
Error connecting: (localhost:58402) failed to connect: [Errno 111] Connection refused, attempt 0
ERROR: tcpserver 17828 : error reading message: End of file
ERROR: tcpserver 17827 : error writing message: Broken pipe
ERROR: tcpserver 39506 : error reading message: End of file
ERROR: tcpserver 24278 : error reading message: End of file
ERROR: tcpserver 39505 : error writing message: Broken pipe
ERROR: tcpserver 24277 : error writing message: Broken pipe
global:Episode#:0        ep_reward:-1149513.8107173594   mean_ep_rew:-1149513.8107173594         best_ep_reward:-1149513.8107173594
ERROR: tcpserver 25508 : error reading message: End of file
Start pos 21 ([0.0, 1.0]), end 65 ([1.0, 1.0])
Starting new episode...
Start pos 21 ([0.0, 1.0]), end 65 ([1.0, 1.0])
Starting new episode...
Start pos 21 ([0.0, 1.0]), end 65 ([1.0, 1.0])
Starting new episode...
Start pos 21 ([0.0, 1.0]), end 65 ([1.0, 1.0])
Starting new episode...
ERROR: tcpserver 0 : error reading message: Operation canceled
ERROR: tcpserver 0 : error reading message: Operation canceled
Error during step, terminating episode early Traceback (most recent call last):
  File "/Hands-On-Intelligent-Agents-with-OpenAI-Gym/ch8/environment/carla_gym/envs/carla/tcp.py", line 67, in write
    self._socket.sendall(header + message)
BrokenPipeError: [Errno 32] Broken pipe

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Hands-On-Intelligent-Agents-with-OpenAI-Gym/ch8/environment/carla_gym/envs/carla_env.py", line 316, in step
    obs = self.step_env(action)
  File "/Hands-On-Intelligent-Agents-with-OpenAI-Gym/ch8/environment/carla_gym/envs/carla_env.py", line 340, in step_env
    self.client.send_control(
  File "/Hands-On-Intelligent-Agents-with-OpenAI-Gym/ch8/environment/carla_gym/envs/carla/client.py", line 145, in send_control
    self._control_client.write(pb_message.SerializeToString())
  File "/Hands-On-Intelligent-Agents-with-OpenAI-Gym/ch8/environment/carla_gym/envs/carla/tcp.py", line 69, in write
    self._reraise_exception_as_tcp_error('failed to write data', exception)
  File "/Hands-On-Intelligent-Agents-with-OpenAI-Gym/ch8/environment/carla_gym/envs/carla/tcp.py", line 97, in _reraise_exception_as_tcp_error
    raise TCPConnectionError('%s%s: %s' % (self._logprefix, message, exception))
environment.carla_gym.envs.carla.tcp.TCPConnectionError: (localhost:21384) failed to write data: [Errno 32] Broken pipe

Clearing Carla server state
ERROR: tcpserver 21382 : error reading message: End of file Initializing new Carla server...
Error during step, terminating episode early Traceback (most recent call last):
  File "/Hands-On-Intelligent-Agents-with-OpenAI-Gym/ch8/environment/carla_gym/envs/carla/tcp.py", line 67, in write
    self._socket.sendall(header + message)
BrokenPipeError: [Errno 32] Broken pipe

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Hands-On-Intelligent-Agents-with-OpenAI-Gym/ch8/environment/carla_gym/envs/carla_env.py", line 316, in step
    obs = self.step_env(action)
  File "/Hands-On-Intelligent-Agents-with-OpenAI-Gym/ch8/environment/carla_gym/envs/carla_env.py", line 340, in step_env
    self.client.send_control(
  File "/Hands-On-Intelligent-Agents-with-OpenAI-Gym/ch8/environment/carla_gym/envs/carla/client.py", line 145, in send_control
    self._control_client.write(pb_message.SerializeToString())
  File "/Hands-On-Intelligent-Agents-with-OpenAI-Gym/ch8/environment/carla_gym/envs/carla/tcp.py", line 69, in write
    self._reraise_exception_as_tcp_error('failed to write data', exception)
  File "/Hands-On-Intelligent-Agents-with-OpenAI-Gym/ch8/environment/carla_gym/envs/carla/tcp.py", line 97, in _reraise_exception_as_tcp_error
    raise TCPConnectionError('%s%s: %s' % (self._logprefix, message, exception))
environment.carla_gym.envs.carla.tcp.TCPConnectionError: (localhost:44330) failed to write data: [Errno 32] Broken pipe

Clearing Carla server state
ERROR: tcpserver 44328 : error reading message: End of file
Initializing new Carla server...
ERROR: tcpserver 24278 : error reading message: End of file
ERROR: tcpserver 39506 : error reading message: End of file
ERROR: tcpserver 39505 : error writing message: Broken pipe
ERROR: tcpserver 17828 : error reading message: End of file
ERROR: tcpserver 17827 : error writing message: Broken pipe
Error connecting: (localhost:58402) failed to connect: [Errno 111] Connection refused, attempt 1
Start pos 21 ([0.0, 1.0]), end 65 ([1.0, 1.0])
Starting new episode...
global:Episode#:1        ep_reward:-416539.880988593     mean_ep_rew:-783026.8458529762  best_ep_reward:-416539.880988593
ERROR: tcpserver 25508 : error reading message: End of file
Start pos 21 ([0.0, 1.0]), end 65 ([1.0, 1.0])
Starting new episode...
ERROR: tcpserver 53828 : error reading message: End of file
Start pos 21 ([-1.0, 2.0]), end 65 ([0.0, 1.0])
Starting new episode...
Start pos 21 ([-1.0, 2.0]), end 65 ([0.0, 1.0])
Starting new episode...
Start pos 21 ([0.0, 1.0]), end 65 ([1.0, 1.0])
Starting new episode...
Start pos 21 ([0.0, 1.0]), end 65 ([1.0, 1.0])
Starting new episode...
Start pos 21 ([0.0, 1.0]), end 65 ([1.0, 1.0])
Starting new episode...
ERROR: tcpserver 24278 : error reading message: End of file
ERROR: tcpserver 24277 : error writing message: Broken pipe
ERROR: tcpserver 39506 : error reading message: End of file
ERROR: tcpserver 39505 : error writing message: Broken pipe
Error connecting: (localhost:58402) failed to connect: [Errno 111] Connection refused, attempt 2
ERROR: tcpserver 19605 : error reading message: End of file
ERROR: tcpserver 19604 : error writing message: Broken pipe
Start pos 21 ([0.0, 1.0]), end 65 ([1.0, 1.0])
Starting new episode...
ERROR: tcpserver 0 : error reading message: Operation canceled
global:Episode#:2        ep_reward:-350793.309871722     mean_ep_rew:-638949.0005258914  best_ep_reward:-350793.309871722
ERROR: tcpserver 25508 : error reading message: End of file
ERROR: tcpserver 17828 : error reading message: End of file
Start pos 21 ([0.0, 1.0]), end 65 ([1.0, 1.0])
Starting new episode...
ERROR: tcpserver 53828 : error reading message: End of file
Start pos 21 ([0.0, 1.0]), end 65 ([1.0, 1.0])
Starting new episode...
Start pos 21 ([0.0, 1.0]), end 65 ([1.0, 1.0])
Starting new episode...
Start pos 21 ([0.0, 1.0]), end 65 ([1.0, 1.0])
Starting new episode...
Error during step, terminating episode early Traceback (most recent call last):
  File "/Hands-On-Intelligent-Agents-with-OpenAI-Gym/ch8/environment/carla_gym/envs/carla/tcp.py", line 67, in write
    self._socket.sendall(header + message)
BrokenPipeError: [Errno 32] Broken pipe

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Hands-On-Intelligent-Agents-with-OpenAI-Gym/ch8/environment/carla_gym/envs/carla_env.py", line 316, in step
    obs = self.step_env(action)
  File "/Hands-On-Intelligent-Agents-with-OpenAI-Gym/ch8/environment/carla_gym/envs/carla_env.py", line 340, in step_env
    self.client.send_control(
  File "/Hands-On-Intelligent-Agents-with-OpenAI-Gym/ch8/environment/carla_gym/envs/carla/client.py", line 145, in send_control
    self._control_client.write(pb_message.SerializeToString())
  File "/Hands-On-Intelligent-Agents-with-OpenAI-Gym/ch8/environment/carla_gym/envs/carla/tcp.py", line 69, in write
    self._reraise_exception_as_tcp_error('failed to write data', exception)
  File "/Hands-On-Intelligent-Agents-with-OpenAI-Gym/ch8/environment/carla_gym/envs/carla/tcp.py", line 97, in _reraise_exception_as_tcp_error
    raise TCPConnectionError('%s%s: %s' % (self._logprefix, message, exception))
environment.carla_gym.envs.carla.tcp.TCPConnectionError: (localhost:48876) failed to write data: [Errno 32] Broken pipe

Clearing Carla server state
ERROR: tcpserver Initializing new Carla server...
Start pos 21 ([0.0, 1.0]), end 65 ([1.0, 1.0])
Starting new episode...
Error connecting: (localhost:58402) failed to connect: [Errno 111] Connection refused, attempt 3
ERROR: tcpserver 24278 : error reading message: End of file
Error during reset: Traceback (most recent call last):
  File "/Hands-On-Intelligent-Agents-with-OpenAI-Gym/ch8/environment/carla_gym/envs/carla_env.py", line 224, in reset
    return self.reset_env()
  File "/Hands-On-Intelligent-Agents-with-OpenAI-Gym/ch8/environment/carla_gym/envs/carla_env.py", line 272, in reset_env
    scene = self.client.load_settings(settings)
  File "/Hands-On-Intelligent-Agents-with-OpenAI-Gym/ch8/environment/carla_gym/envs/carla/client.py", line 75, in load_settings
    return self._request_new_episode(carla_settings)
  File "/Hands-On-Intelligent-Agents-with-OpenAI-Gym/ch8/environment/carla_gym/envs/carla/client.py", line 158, in _request_new_episode
    self._world_client.write(pb_message.SerializeToString())
  File "/Hands-On-Intelligent-Agents-with-OpenAI-Gym/ch8/environment/carla_gym/envs/carla/tcp.py", line 64, in write
    raise TCPConnectionError(self._logprefix + 'not connected')
environment.carla_gym.envs.carla.tcp.TCPConnectionError: (localhost:58402) not connected

Clearing Carla server state
Initializing new Carla server...
global:Episode#:3        ep_reward:-319133.8765342036    mean_ep_rew:-558995.2195279695  best_ep_reward:-319133.8765342036
ERROR: tcpserver 19605 : error reading message: End of file
ERROR: tcpserver 25508 : error reading message: End of file
Error connecting: (localhost:42652) failed to connect: [Errno 111] Connection refused, attempt 0
ERROR: tcpserver 25507 : error writing message: Broken pipe
ERROR: tcpserver 39506 : error reading message: End of file
Start pos 21 ([0.0, 1.0]), end 65 ([1.0, 1.0])
Starting new episode...
Start pos 21 ([-1.0, 2.0]), end 65 ([0.0, 1.0])
Starting new episode...
Start pos 21 ([0.0, 1.0]), end 65 ([1.0, 1.0])
Starting new episode...
ERROR: tcpserver 17828 : error reading message: End of file
Start pos 21 ([0.0, 1.0]), end 65 ([1.0, 1.0])
Starting new episode...
ERROR: tcpserver 53828 : error reading message: End of file
Start pos 21 ([0.0, 1.0]), end 65 ([1.0, 1.0])
Starting new episode...
Start pos 21 ([-1.0, 2.0]), end 65 ([0.0, 1.0])
Starting new episode...
Start pos 21 ([0.0, 1.0]), end 65 ([1.0, 1.0])
Starting new episode...
Start pos 21 ([0.0, 1.0]), end 65 ([1.0, 1.0])
Starting new episode...
ERROR: tcpserver 39506 : error reading message: End of file
ERROR: tcpserver 39505 : error writing message: Broken pipe
ERROR: tcpserver 42654 : error reading message: End of file
ERROR: tcpserver 24278 : error reading message: End of file
ERROR: tcpserver 42653 : error writing message: Broken pipe
ERROR: tcpserver 19605 : error reading message: End of file
global:Episode#:4        ep_reward:-732070.2213848312    mean_ep_rew:-593610.2198993418  best_ep_reward:-319133.8765342036
ERROR: tcpserver 25508 : error reading message: End of file
ERROR: tcpserver 25507 : error writing message: Broken pipe
Start pos 21 ([0.0, 1.0]), end 65 ([1.0, 1.0])
Starting new episode...
Start pos 21 ([0.0, 1.0]), end 65 ([1.0, 1.0])
Starting new episode...
Start pos 21 ([0.0, 1.0]), end 65 ([1.0, 1.0])
Starting new episode...
ERROR: tcpserver 0 : error reading message: Operation canceled
Start pos 21 ([0.0, 1.0]), end 65 ([1.0, 1.0])
Starting new episode...
Start pos 21 ([0.0, 1.0]), end 65 ([1.0, 1.0])
Starting new episode...
ERROR: tcpserver 53828 : error reading message: End of file
ERROR: tcpserver 53827 : error writing message: Broken pipe
ERROR: tcpserver 17828 : error reading message: End of file
Error during step, terminating episode early Traceback (most recent call last):
  File "/Hands-On-Intelligent-Agents-with-OpenAI-Gym/ch8/environment/carla_gym/envs/carla/tcp.py", line 67, in write
    self._socket.sendall(header + message)
BrokenPipeError: [Errno 32] Broken pipe

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Hands-On-Intelligent-Agents-with-OpenAI-Gym/ch8/environment/carla_gym/envs/carla_env.py", line 316, in step
    obs = self.step_env(action)
  File "/Hands-On-Intelligent-Agents-with-OpenAI-Gym/ch8/environment/carla_gym/envs/carla_env.py", line 340, in step_env
    self.client.send_control(
  File "/Hands-On-Intelligent-Agents-with-OpenAI-Gym/ch8/environment/carla_gym/envs/carla/client.py", line 145, in send_control
    self._control_client.write(pb_message.SerializeToString())
  File "/Hands-On-Intelligent-Agents-with-OpenAI-Gym/ch8/environment/carla_gym/envs/carla/tcp.py", line 69, in write
    self._reraise_exception_as_tcp_error('failed to write data', exception)
  File "/Hands-On-Intelligent-Agents-with-OpenAI-Gym/ch8/environment/carla_gym/envs/carla/tcp.py", line 97, in _reraise_exception_as_tcp_error
    raise TCPConnectionError('%s%s: %s' % (self._logprefix, message, exception))
environment.carla_gym.envs.carla.tcp.TCPConnectionError: (localhost:45909) failed to write data: [Errno 32] Broken pipe

Clearing Carla server state
ERROR: tcpserver 45907 : error reading message: End of file
ERROR: tcpserver 17827 : error writing message: Broken pipe
Initializing new Carla server...

Thank you for explaining why this problem could happen.
Besides, do you know any ways to let me use carla on multiple GPUs? I have four GPUs now, but during training, the process could only work on a single GPU.

Hi @Panshark , I missed seeing this message until now. Sorry for the delayed response.
One way to run CARLA on multiple GPUs for training is using MACAD-Gym which finds the GPU with less utilization and launches CARLA server process on that GPU. You can instantiate multiple environments to utilize all of your GPUs.
Hope that helps.

Hi @Panshark , I missed seeing this message until now. Sorry for the delayed response.
One way to run CARLA on multiple GPUs for training is using MACAD-Gym which finds the GPU with less utilization and launches CARLA server process on that GPU. You can instantiate multiple environments to utilize all of your GPUs. Hope that helps.

Hi @praveen-palanisamy ,
Thank you for replying. I tried to use
import os
os.environ["CUDA_VISIBLE_DEVICES"] = "1"

to let CARLA only use GPU No.1 but failed.
Whenever I use CARLA with MACAD-GYM, it always uses the No.0 GPU. The only thing I can do is put my training program on the other GPUs and retain more resources on the No.0 GPU. Do you know how to fix this CARLA simulator always runs at default GPU issue?

The environment variable you set in the script may not be in use by the terminal that launches the CARLA server process.
To isolate the issue, please try to run the CARLA server binary directly using this command from a bash terminal: CUDA_VISIBLE_DEVICES=1 ./CarlaUE4.sh and see/share the output of watch nvidia-smi to identify the Nvidia GPU utilization.
You can also try to force the GPU device ID used for rendering using the SDL flag: CUDA_VISIBLE_DEVICE=1 SDL_HINT_CUDA_DEVICE=1 ./CarlaUE4.sh -RenderOffscreen.
Add -RenderOffscreen at the end of the above commands if you want to render offscreen (no display) like this:  CUDA_VISIBLE_DEVICE=1 SDL_HINT_CUDA_DEVICE=1 ./CarlaUE4.sh -RenderOffscreen`
If you are using MACAD-Gym, and have multiple GPUs and vglrun setup on your system, you should see: "Running sim servers in headless/multi-GPU mode" in your MACAD-Gym log file and the CARLA server process should be launched on a less utilized GPU.
Please confirm with the logfile.

@Panshark : I hope the above answer helped to resolve your issue. Since there hasn't been a follow-up in a long while, I am closing this issue. Please feel free to re-open or create a new discussion if you have follow-up questions.


Domain not included in url

Images do not work on platforms like twitter without a full url

:tada: This issue has been resolved in version 1.3.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:

:tada: This issue has been resolved in version 1.0.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Daily builds are supposed to be older than 1 day?

In bcinsider.azurecr.io/bcsandbox-master , which is supossed to be updated regularly, we still dont have new event definitions that are marked to ship-in-next-update.
Is this normal? When we can expect this docker to be updated?
We are in lot of development that need those events and it would be nice to have a docker updated.

https://freddysblog.com/2019/05/29/business-central-fall-2019-preview-availability
I expect bcsandbox-master to restart within a week.

Thx for the info.
I´m closing the issue.
Kind regards.


[docs] update getting-started to work with cosign 2.1

Summary

Trying to follow the getting started guide is currently rather hard due to the cosign version used is rather old.
To make it easier I updated the documentation.
Release Note

NONE
Documentation

I have tested all the commands with cosign version 2.1.0 I chose not to update the example output, personally I think it's close enough, and I think people will understand.
I also tried to run with cosign 2.2.0, but then I got the following error
Handling connection for 8080
Error: signing registry.local:5001/sigstore/demo@sha256:95bb686784c1d0deafc656486ae562e0420ab2b9a2f4c307a6571d401ce00e3e: [POST /api/v1/log/entries][422] createLogEntry default  &{Code:422 Message:invalid kind value: "dsse"}
main.go:74: error during command execution: signing registry.local:5001/sigstore/demo@sha256:95bb686784c1d0deafc656486ae562e0420ab2b9a2f4c307a6571d401ce00e3e: [POST /api/v1/log/entries][422] createLogEntry default  &{Code:422 Message:invalid kind value: "dsse"}

I'm 90% sure this is due to https://github.com/sigstore/cosign/pull/3113.
I think this PR is better than the current state, but ideally it should work with 2.2.0.
Furthermore, I feel that I'm a bit out of depth trying to debug why this happens, but looking at the PR it seems to be a change in the way of working.
If anyone with more knowledge about the dsse change, please share.

Thanks for the fix @bobcallaway , I was at PTO so that why I was so slow to update.
Is there anything else needed for the PR you feel?


odoc doesn't respect stop comments

For example in uuidm we have this stop comment:
https://github.com/dbuenzli/uuidm/blob/master/src/uuidm.mli#L38-L41
create still shows up in odoc's API doc with dubious C multiline start comments.
opam install uuidm && opkg ocamldoc uuidm && opkg odoc uuidm && opkg doc -c uuidm

The problem in this case is that doc-ock expects stop comments to be floating comments whereas in your example the compiler has attached the comments to create.
I feel like we should keep the implementation of odoc as is and you should add some blank lines in your code so they are not attached to the node.

I think this is going to be a problem in practice. The specification really says that everything between two (**/**) is dropped. It feels a little brittle to ask people to add whitespace for this to work.

It feels brittle

Really? Properly handling these comments when they are attached to a node involves making some relatively ugly Location-based decisions.
Just asking people to add a newline here and there doesn't feel "brittle" to me and we already asked them to add such newlines in the past, e.g. to disambiguate things like:
val foo : 'a -> 'b
(** where do I attach? *)
val bar : 'a -> 'b

so it doesn't seem too crazy to make the same request in this case.
@lpw25 any opinion?

Really?

Yes. Your example is quite different. As you can read in the specification I linked to there's no ambiguity in a (**/**) comment.
My take on this is that the comment lifting is wrongly done upstream. (**/**) should never get attached to a value specification.

I agree on both points. It's just that I don't feel like the compiler is going to be "fixed" on that point (but @lpw25 might contradict me on this?) and I'm not really keen on introducing shitty "detachment" decisions based on locations in doc-ock.
That being said I don't mind submitting a PR on the compiler just to see how it goes.

I'm not really keen on introducing shitty "detachment" decisions based on locations in doc-ock.

Sure it doesn't seem to be the right place to do this.

Excellent @trefis and thanks to @damiendoligez for merging this into 4.04


Repeating overline syntax with Clearspeak

I am not sure if this is expected behaviour or not, but logging here just in case. Given the following MathML:
<math xmlns="http://www.w3.org/1998/Math/MathML" display="block">
  <mn>5.178</mn>
  <mover>
    <mn>6</mn>
    <mo accent="false">¯<!-- ¯ --></mo>
  </mover>
</math>

The Mathspeak rules output:
./bin/sre -i input.xml -d mathspeak -t default -c en
-> 5.178 ModifyingAbove 6 With bar

However with Clearspeak enabled, the output is this:
./bin/sre -i input.xml -d clearspeak -t default -c en
-> the repeating decimal 5 point 1 7 8 followed by repeating digit 6

Is the repeating decimal 5 supposed to be there as an indicator that there are going to be repeating digits in the number? Or should it be simply read as: 5 point 1 7 8 followed by repeating digit 6?

Hi Ryan, sorry it took me a while to get back to this.
The reading is indeed correct. Have look at the Rules and Preferences guide page 61. The rule there states:
the repeating decimal [number] with repeating digit [digit under bar]
The examples however (e.g. in document ClearSpeakExamples_Part 3_Adornments.docx) all use
followed by
For example,
The repeating decimal 2 point followed by repeating digits 1 3 4
I have chosen to implement the latter. But in hindsight the original rule might actually make it a bit shorter.

Perfect, thanks for clearing that up Volker!


Executable code preview “Unable to decode transaction data”

Executable code preview “Unable to decode transaction data”
Can we have more context around this message?
Something isn’t working and whether or not it is our app or just a slow network, we should clearly communicate whats going on as much as possible

Originally posted by @xraystyle1980 in https://github.com/decentdao/fractal-interface/issues/1578#issuecomment-2101312887

Marking as Medium task as it might take some thought on how to capture what the problem is.

The problem is that our frontend was unable to gather the necessary information to provide more context to the user (function name, parameter types, parameter inputs) for the given transaction.
I understand that there might be multiple reasons for that... but does the user really care? Doubt it.
@decentdao/designers @tomstuart123 can you please whip up some better copy (and design?) for us here?

Agreed, for now lets not try and be too specific or detailed with the error message
@xraystyle1980 / @nicolaus-sherrill how about we lean into something just a bit more empathetic/human sounding /
@adamgall is there somewhere we can point them where they might be able to decode the data? I don't think so but just checking

I feel you @adamgall, thanks
So ye @xraystyle1980 , I think the only upgrade we could explore for now is more human (less cold language) but not urgent if you / @nicolaus-sherrill don't have brainspace
Lets pick up personalised errors/recommendations later

@adamgall As far as better design, if that is an error, we should use a red-colored error state pattern for better visual feedback. (Same colors as the other error messages, but please let me know if a visual is needed.)
For copy, idk, how about "We are unable to retrieve transaction details at this time."
(Is there any way to refresh the data without refreshing the page???)

@adamgall As far as better design, if that is an error, we should use a red-colored error state pattern for better visual feedback. (Same colors as the other error messages, but please let me know if a visual is needed.)
For copy, idk, how about "We are unable to retrieve transaction details at this time."
(Is there any way to refresh the data without refreshing the page???)

@nicolaus-sherrill Well idk if it's really an "error"... there are valid reasons as to why we can't show decoded transaction data to the user which are not an error or anything we can do about.
I like the copy suggestion.
We are already trying to refresh this data in the background.
@decentdao/designers in addition to the copy updates suggested by @nicolaus-sherrill , are there are design changes to this "We are unable to retrieve transaction details at this time." component you'd like to implement?

Once the Safe API Kit upgrade happens, split messaging to one of two possible scenarios:

Safe API errors out.
Data retrieved could not be made human readable.

Closed via https://github.com/decentdao/fractal-interface/pull/1760


temporal chart: scroll to tweaks

Currently housed in the scrollTo on line 133 of the graphicX.js file. Then being called inside the scrollToTheme function on line 255. You probably can just tweak the offest values ever so slightly so that the scroll behavoir lines up with the enterview behavior housed in the runEnterview function on line 187 and called on line 362.


Update management-groups-roles.md

Edited sentence references an administrator as being "it"...
"Once the Azure roles have been assigned to the users, the tenant administrator should remove itself from the user access administrator role."
Themself would be a better choice of reflexive pronoun and is considered a proper non-gender specific term for humans.  If the preference is to indicate all living beings are together in this world and we are all things, no changes are required.

@jellers : Thanks for your contribution! The author(s) have been notified to review your proposed change.

@jellers Thanks for this update.
@memildin Would it be accurate to say that "the tenant administrator should remove their account from the user access administrator role"? It would smooth out the grammar, if the administrator is in fact removing their account from the role. Thanks.

@ktoliver it seems you now have conflicts here to resolve.

I'm not finding the file in the repository. I believe that the content has been reorganized, so this pull request can't be merged in.
@jellers If the update still needs to be made, could you create a new pull request? We'll close this one. Thanks.


Fix the race in the test

The test was inherently racy and I saw it flake more than once (e.g. https://prow.knative.dev/view/gs/knative-prow/pr-logs/pull/knative_pkg/1526/pull-knative-pkg-unit-tests/1286086935722332160)
So this fixes it.
Also other minor nits around the test to make it more readable.
/assign mattmoor @yanweiguo

Thanks for the fix.


How to obtain the camera parameters?

Thank you very much for your work. I would like to ask, given a photo, how can I obtain the associated camera parameters?

It's in the README
https://github.com/SizheAn/PanoHead/tree/main#obtaining-camera-pose-and-cropping-the-images

It's in the README https://github.com/SizheAn/PanoHead/tree/main#obtaining-camera-pose-and-cropping-the-images

I have a small question, if 3DDFA use face to confirm camera angle, how can it process the khair dataset

Can you provide a link to / attach examples from the Khair dataset?


Add png logo version

Here is my small contribution 😊
I update the logo to the .png version to make dark mode look good 😎
Thanks for inspiring us to do cool stuff like this one ❤

☀ Light mode
🌙 Dark mode

The best ❤ .


Added clear-search button on user-list search bar.

A clear-search option to clear the user-list searchbox has been added.
This feature was present in the main searchbar but absent elsewhere.
Fix a part of #3716.

Automated message from Dropbox CLA bot
@jajodiaraghav, it looks like you've already signed the Dropbox CLA. Thanks!

This has some rendering issues in Chrome on Linux:

Fixed :)

@brockwhittaker can you review?

This looks pretty solid, but I have a few comments pertaining to variable/attribute names.

@showell Done :)

@timabbott, After the last round of changes from @jajodiaraghav, I am very happy with the code here, but I wonder if we should have Brock or somebody do another round of testing before merging.
Thanks for the quick turnaround, @jajodiaraghav !

I just played with this, and posted one comment on the CSS organization, otherwise this looks good to me!  I'm not 100% confident that we've picked the right behavior (I could imagine only showing the 'x' if there is text, regardless of focus), but I think we'll need to experience it in action to decide.
So @jajodiaraghav I can merge once you address that CSS comment.

@timabbott Updated :)

Merged, thanks @jajodiaraghav!!
If you're planning to work on the other search bars, it might eventually make sense to move some of the CSS into components.css with a nice reusable component name.