WinUI 3.0 Alpha: app certification kit fails

Describe the bug
App certification kit fails to validate an app, that uses WebView2.
Logs attached: AppCertKitValidationResult.zip
Steps to reproduce the bug

Steps to reproduce the behavior:

Create blank WinUI app
Add WebView2 to MainPage, set UriSource to "https://bing.com"
Right click on the project -> Publish -> Create App Packages...
Enable x86 and x64 targets
In the dialog, ensure you enabled "Generate artifacts to validate the app with the Windows App Certification Kit"
Wait for build to complete and launch certification

Expected behavior
Certification should pass.
Version Info

NuGet package version:
Microsoft.UI.Xaml 3.0.0-alpha.200210.0
Microsoft.Web.WebView2 0.8.355

Windows 10 version
Saw the problem?

Insider Build (xxxxx)

November 2019 Update (18363)
Yes

May 2019 Update (18362)

October 2018 Update (17763)

April 2018 Update (17134)

Fall Creators Update (16299)

Creators Update (15063)

Device form factor
Saw the problem?

Desktop
Yes

Mobile

Xbox

Surface Hub

IoT

@lostmsu This is an alpha release to get early feedback and currently we do not support publishing production apps with the store as mentioned in the release notes.


Automated UI testing: project creation #495

Description of the PR
Closes #495.
Kotlin is here! This PR contains initial steps for automated end-to-end testing.
That is, creating a new project and setting SDK to OpenJDK 11. Currently, the test only works if
run on clean IDEA.
For UI tests, a new source set "e2e" is introduced. IntelliJ IDEA does not automatically recognize the source set's code as "test" code (green color). For the best development experience, you should change that after updating Gradle (see pic).

(Note: building, testing and other important things should work perfectly without any manual configuring. The only things that the fix above concerns are such as when you "Find Usages" in IDEA, whether the code in e2e source set are considered "production" code or "test" code. This is a bug in Gradle's IDEA plug-in that apparently comes and goes, and JetBrains doesn't consider it that important.)
Testing

unit
integration
manual

- [ ] new unit tests created
- [ ] all unit tests pass

- [ ] new integration tests created
- [ ] all integration tests pass

- [ ] manual testing went well

Have you updated the TESTING.md or other relevant documentation on your branch?

[ ] Yes.
[ ] Not yet. I will do it next.
[x] Not relevant.

@stellatsv I added you as a reviewer because I remembered you had learned some Kotlin. :)

IntelliJ IDEA does not automatically recognize the source set's code as "test" code (green color). For the best development experience, you should change that after updating Gradle (see pic).

for me IDEA did recognize the e2e as sources root due to being located in src, I'd actually suggest the following structure:

I would like to know if the setup could work in a headless env (like GH action build) 🤓

I would like to know if the setup could work in a headless env (like GH action build)

https://github.com/JetBrains/intellij-ui-test-robot/blob/master/ui-test-example/FAQ.md#can-i-run-the-tests-on-github-actions

IntelliJ IDEA does not automatically recognize the source set's code as "test" code (green color). For the best development experience, you should change that after updating Gradle (see pic).

for me IDEA did recognize the e2e as sources root due to being located in src,

To avoid any confusion, let's be precise: IDEA should (and at least it did for me) recognize src/e2e/kotlin as sources root. That is not, however, because it is located in src (src/test/java is too!) but because of these lines in build.gradle:
sourceSets {
    e2e {
        kotlin.srcDir "$projectDir/src/e2e/kotlin"
    }
}

IDEA simply assumes an included dir is sources root instead of test root (let's be honest, the opposite would be worse).

I'd actually suggest the following structure:

That's how I organized it in the #492. In this time, I thought it would be beneficial to have UI tests in a separate source set (that does not see the production code but handles the plugin UI as a black box). It's apparent, both the approaches have their pros and cons (e.g. I dislike #492's ugly test filters in build.gradle).
But if we keep e2e as a separate source set (i.e. module), how could we tell IDEA that src/e2e/kotlin should be test root instead of sources root?
Here's the answer. Basically, we add idea plugin to Gradle and configure:
idea {
  module {
    testSourceDirs += project.sourceSets.e2e.kotlin.srcDirs
  }
}

Too bad, that doesn't work anymore with the newest IDEA versions (because of a bug?).
As the "fix" above were never meant to be the final solution to the issue, it is possible they're not paying much attention to get it working again. In the future, I hope there are gonna be proper ways to equip Gradle's source sets with meaningful IDEA semantics.

I would like to know if the setup could work in a headless env (like GH action build)

https://github.com/JetBrains/intellij-ui-test-robot/blob/master/ui-test-example/FAQ.md#can-i-run-the-tests-on-github-actions

I suggest we create a new ticket of that because UI tests can be put into operation also even if they were still triggered manually.

regardless of where we put koltin, shall there be a checkstyle for it? @OlliKiljunen

regardless of where we put koltin, shall there be a checkstyle for it? @OlliKiljunen

I guess I should check what happens if I click this notification:

Tests pass now for me as well :tada: We might want to consider removing the JDK-installation from the UI tests, but it's also fine as is :+1:


Azd Command Restructuring

Iterate on Azd Command Restructuring

@rajeshkamal5050 I don't think that this is required anymore. Let's open more granular items as needed.


Add a mentor's name to each work item

It will be great to add a mentor's name to each work item so that the new contributor knows whom to ask for guidance/help.

Related to #50

Thanks @cathyhongzhang!
CLOTributor feeds mostly from information available in the issues (plus some details from the repository). To make something like this work, the mentor name should be part of the issue somehow (maybe as a label?).
At the moment the main CLOTributor's goal is to make it easier to discover and find issues available across CNCF (and other foundations) projects for contributors. But once a contributor finds an interesting issue that suits them, the process continues in GitHub and, once the issue is assigned, CLOTributor stops listing it. So hopefully contributors will find the guidance and help they need on the issue, which has all the work's context and it's probably where all conversations will happen.
But it's possible that eventually CLOTributor will take more responsibilities and in that case we'd definitely need to track both mentors and mentees.
Will close this one for now, please feel free to reopen if needed 🙂


XPSDrvSample

Hi,
I get the following error message when I try and build the "XPSDrv Driver and Filter Sample" project. Followed the instructions in the Readme guide, but I also get exact same error message when Building the built-in "Print Driver v4" template. Not sure if it's something I have to configure within each project or on my machine.
Using Visual Studio 2015 Express.
Inf2Cat Tool Output:
.............................
Signability test failed.
Errors:
22.9.1: xdwscrgb.icc in [colorprofiles] of \xdsmpl.inf is missing or cannot be decompressed from source media.  Please verify all path values specified in SourceDisksNames, SouceDisksFiles, and CopyFiles sections resolve to the actual location of the file, and are expressed in terms relative to the location of the inf.
22.9.1: xdcmykprinter.icc in [colorprofiles] of \xdsmpl.inf is missing or cannot be decompressed from source media.  Please verify all path values specified in SourceDisksNames, SouceDisksFiles, and CopyFiles sections resolve to the actual location of the file, and are expressed in terms relative to the location of the inf.
22.9.1: xdsmpl.ini in [configplugin] of \xdsmpl.inf is missing or cannot be decompressed from source media.  Please verify all path values specified in SourceDisksNames, SouceDisksFiles, and CopyFiles sections resolve to the actual location of the file, and are expressed in terms relative to the location of the inf.
22.9.1: xdsmpl.gpd in [xpsdrvsample] of \xdsmpl.inf is missing or cannot be decompressed from source media.  Please verify all path values specified in SourceDisksNames, SouceDisksFiles, and CopyFiles sections resolve to the actual location of the file, and are expressed in terms relative to the location of the inf.
22.9.1: xdnames.gpd in [xpsdrvsample] of \xdsmpl.inf is missing or cannot be decompressed from source media.  Please verify all path values specified in SourceDisksNames, SouceDisksFiles, and CopyFiles sections resolve to the actual location of the file, and are expressed in terms relative to the location of the inf.
22.9.1: xdwmark.gpd in [xpsdrvsample] of \xdsmpl.inf is missing or cannot be decompressed from source media.  Please verify all path values specified in SourceDisksNames, SouceDisksFiles, and CopyFiles sections resolve to the actual location of the file, and are expressed in terms relative to the location of the inf.
22.9.1: xdbook.gpd in [xpsdrvsample] of \xdsmpl.inf is missing or cannot be decompressed from source media.  Please verify all path values specified in SourceDisksNames, SouceDisksFiles, and CopyFiles sections resolve to the actual location of the file, and are expressed in terms relative to the location of the inf.
22.9.1: xdcolman.gpd in [xpsdrvsample] of \xdsmpl.inf is missing or cannot be decompressed from source media.  Please verify all path values specified in SourceDisksNames, SouceDisksFiles, and CopyFiles sections resolve to the actual location of the file, and are expressed in terms relative to the location of the inf.
22.9.1: xdnup.gpd in [xpsdrvsample] of \xdsmpl.inf is missing or cannot be decompressed from source media.  Please verify all path values specified in SourceDisksNames, SouceDisksFiles, and CopyFiles sections resolve to the actual location of the file, and are expressed in terms relative to the location of the inf.
22.9.1: xdpgscl.gpd in [xpsdrvsample] of \xdsmpl.inf is missing or cannot be decompressed from source media.  Please verify all path values specified in SourceDisksNames, SouceDisksFiles, and CopyFiles sections resolve to the actual location of the file, and are expressed in terms relative to the location of the inf.
22.9.1: xdsmpl-pipelineconfig.xml in [xpsdrvsample] of \xdsmpl.inf is missing or cannot be decompressed from source media.  Please verify all path values specified in SourceDisksNames, SouceDisksFiles, and CopyFiles sections resolve to the actual location of the file, and are expressed in terms relative to the location of the inf.
Warnings:
22.9.3: Missing hardware ID (cannot have just compatible ID) in line xpsdrv sample driver=install_xdsmpl_filters_pre_vista in section [standard.ntx86] of \xdsmpl.inf
22.9.3: Missing hardware ID (cannot have just compatible ID) in line xpsdrv sample driver=install_xdsmpl_filters_pre_vista in section [standard.ntia64] of \xdsmpl.inf
22.9.3: Missing hardware ID (cannot have just compatible ID) in line xpsdrv sample driver=install_xdsmpl_filters_pre_vista in section [standard.ntamd64] of \xdsmpl.inf
22.9.3: Missing hardware ID (cannot have just compatible ID) in line xpsdrv sample driver=install_xdsmpl_filters_vista in section [standard.ntx86.6.0] of \xdsmpl.inf
22.9.3: Missing hardware ID (cannot have just compatible ID) in line xpsdrv sample driver=install_xdsmpl_filters_vista in section [standard.ntia64.6.0] of \xdsmpl.inf
22.9.3: Missing hardware ID (cannot have just compatible ID) in line xpsdrv sample driver=install_xdsmpl_filters_vista in section [standard.ntamd64.6.0] of \xdsmpl.inf

aaaaa

Same as #123


i18n(ko-KR): create link-cards.mdx

Description

create link-cards.mdx

Lunaria Status Overview
🌕 This pull request will trigger status changes.

Learn more
By default, every PR changing files present in the Lunaria configuration's files property will be considered and trigger status changes accordingly.
You can change this by adding one of the keywords present in the ignoreKeywords property in your Lunaria configuration file in the PR's title (ignoring all files) or by including a tracker directive in the merged commit's description.

Tracked Files

Locale
File
Note

ko
components/link-cards.mdx
Localization added, will be marked as complete.

Warnings reference

Icon
Description

🔄
The source for this localization has been updated since the creation of this pull request, make sure all changes in the source have been applied.

When https://github.com/withastro/starlight/pull/2353 is successfully merged, I expect any errors that occur in this PR to be resolved.


Added 'see also' for transformDatabaseUser

It took me like half an hour of digging to figure out why validateUser() was returning an object with { objectId: string } instead of the entire user object. Added a link hint to help others who may be in the same shoes in the future.

I'm guessing you're using your own adapter? Personally, I think either the user attributes page or the username example should make that clear. My worry is that if we add a 'see also' link here, we have to add it everywhere.

Yes, I'm using my own adapter.
Just realized now that if I clicked on the User link under validateUser(), it does in fact mention transformDatabaseUser. So yeah, I see that adding a 'See also' inside validateUser is not necessary. I'll close the PR.


Allow direction "vertical-reverse"

This is a (multiple allowed):

[ ] bug

[x] enhancement

[ ] feature-discussion (RFC)

Swiper Version: 3.4.2

Platform/Target and Browser Versions: Windows, CHROME

I want to have a slide with the first slide at the bottom and others on top. "slideToClickedSlide" should then work as one would expect.
I tried setting:
    flex-direction: column-reverse;

but then duplicated slides are on the wrong side.

BTW: This is one of the most commented feature requests in slick: https://github.com/kenwheeler/slick/issues/818

Did you tried this ?
      slidesToScroll: -1,
Slick carousel right to left

Issue is closed because of outdated/irrelevant/not actual/needed more information/inactivity.
If this issue is still actual and reproducible for latest version of Swiper, please create new issue and fill the issue template correctly:

Clearly describe the issue including steps to reproduce when it is a bug.
Make sure you fill in the earliest version that you know has the issue.
Provide live link or JSFiddle/Codepen or website with issue


stable/postgresql] issue title

Describe the bug
A clear and concise description of what the bug is.
Because my cluster cannot access internet and helm is unavailable, I have to fill in templates and run 'kubectl apply -f' to deploy the postgresql. But  rejecting connections.
Version of Helm and Kubernetes:
Which chart:
postgresql
What happened:
In log of master, it shows  Role "repl_user" and "stolon" does not exist. "repl_user" is POSTGRES_REPLICATION_USER and "stolon" is POSTGRES_USER
'''
kubectl logs -n goharbor postgresql-master-0
12:28:02.43 Welcome to the Bitnami postgresql container
12:28:02.43 Subscribe to project updates by watching https://github.com/bitnami/bitnami-docker-postgresql
12:28:02.44 Submit issues and feature requests at https://github.com/bitnami/bitnami-docker-postgresql/issues
12:28:02.44 Send us your feedback at<EMAIL_ADDRESS>12:28:02.44
12:28:02.47 INFO  ==> ** Starting PostgreSQL setup **
12:28:02.57 INFO  ==> Validating settings in POSTGRESQL_* env vars..
12:28:02.58 INFO  ==> Initializing PostgreSQL database...
12:28:02.60 INFO  ==> postgresql.conf file not detected. Generating it...
12:28:02.62 INFO  ==> pg_hba.conf file not detected. Generating it...
12:28:02.64 INFO  ==> Deploying PostgreSQL with persisted data...
12:28:02.64 INFO  ==> Cleaning stale postmaster.pid file
12:28:02.79 INFO  ==> Configuring replication parameters
12:28:02.85 INFO  ==> Configuring fsync
12:28:02.87 INFO  ==> Loading custom scripts...
12:28:02.88 INFO  ==> Enabling remote connections
12:28:02.90 INFO  ==> Stopping PostgreSQL...
12:28:02.91 INFO  ==> ** PostgreSQL setup finished! **
12:28:03.01 INFO  ==> ** Starting PostgreSQL **
2019-08-29 12:28:03.249 GMT [1] LOG:  listening on IPv4 address "<IP_ADDRESS>", port 5432
2019-08-29 12:28:03.249 GMT [1] LOG:  listening on IPv6 address "::", port 5432
2019-08-29 12:28:03.253 GMT [1] LOG:  listening on Unix socket "/tmp/.s.PGSQL.5432"
2019-08-29 12:28:04.659 GMT [154] LOG:  database system was interrupted; last known up at 2019-08-29 12:27:22 GMT
2019-08-29 12:28:12.040 GMT [163] FATAL:  the database system is starting up
2019-08-29 12:28:22.080 GMT [172] FATAL:  the database system is starting up
2019-08-29 12:28:32.050 GMT [180] FATAL:  the database system is starting up
2019-08-29 12:29:23.821 GMT [154] LOG:  database system was not properly shut down; automatic recovery in progress
2019-08-29 12:29:24.135 GMT [154] LOG:  redo starts at 0/1649C98
2019-08-29 12:29:24.136 GMT [154] LOG:  invalid record length at 0/1649D40: wanted 24, got 0
2019-08-29 12:29:24.136 GMT [154] LOG:  redo done at 0/1649CD0
2019-08-29 12:29:24.635 GMT [253] FATAL:  the database system is starting up
2019-08-29 12:29:25.652 GMT [254] FATAL:  the database system is starting up
2019-08-29 12:29:26.644 GMT [1] LOG:  database system is ready to accept connections
2019-08-29 12:29:27.078 GMT [261] FATAL:  password authentication failed for user "repl_user"
2019-08-29 12:29:27.078 GMT [261] DETAIL:  Role "repl_user" does not exist.
Connection matched pg_hba.conf line 1: "host     all             all             <IP_ADDRESS>/0               md5"
2019-08-29 12:29:27.299 GMT [262] FATAL:  password authentication failed for user "repl_user"
2019-08-29 12:29:27.299 GMT [262] DETAIL:  Role "repl_user" does not exist.
Connection matched pg_hba.conf line 4: "host      replication     all             <IP_ADDRESS>/0               md5"
2019-08-29 12:29:28.756 GMT [263] FATAL:  password authentication failed for user "repl_user"
2019-08-29 12:29:28.756 GMT [263] DETAIL:  Role "repl_user" does not exist.
Connection matched pg_hba.conf line 4: "host      replication     all             <IP_ADDRESS>/0               md5"
2019-08-29 12:29:29.789 GMT [264] FATAL:  password authentication failed for user "repl_user"
2019-08-29 12:29:29.789 GMT [264] DETAIL:  Role "repl_user" does not exist.
Connection matched pg_hba.conf line 4: "host      replication     all             <IP_ADDRESS>/0               md5"
2019-08-29 12:29:32.056 GMT [273] FATAL:  password authentication failed for user "stolon"
2019-08-29 12:29:32.056 GMT [273] DETAIL:  Role "stolon" does not exist.
Connection matched pg_hba.conf line 1: "host     all             all             <IP_ADDRESS>/0               md5"
'''
What you expected to happen:
How to reproduce it (as minimally and precisely as possible):
Anything else we need to know:

But I have config them in yaml files
'''
apiVersion: apps/v1beta2
kind: StatefulSet
metadata:
name: cambricon-postgresql-master
labels:
app: postgresql
release: "cambricon"
spec:
serviceName: cambricon-postgresql-headless
replicas: 1
updateStrategy:
type: RollingUpdate
selector:
matchLabels:
app: postgresql
release: "cambricon"
role: master
template:
metadata:
name: cambricon-postgresql
labels:
app: postgresql
release: "cambricon"
role: master
spec:
securityContext:
fsGroup: 1001
containers:
- name: cambricon-postgresql
image: docker.cambricon.com/library/goharbor/postgresql:11.4.0-debian-9-r12
imagePullPolicy: "IfNotPresent"
resources:
requests:
cpu: 250m
memory: 256Mi
    securityContext:
      runAsUser: 1001
    env:
    - name: BITNAMI_DEBUG
      value: "false"
    - name: POSTGRESQL_PORT_NUMBER
      value: "5432"
    - name: PGDATA
      value: "/bitnami/postgresql/data"
    - name: POSTGRES_REPLICATION_MODE
      value: "master"
    - name: POSTGRES_REPLICATION_USER
      value: "repl_user"
    - name: POSTGRES_REPLICATION_PASSWORD
      valueFrom:
        secretKeyRef:
          name: cambricon-postgresql
          key: postgresql-replication-password
    - name: POSTGRES_SYNCHRONOUS_COMMIT_MODE
      value: "on"
    - name: POSTGRES_NUM_SYNCHRONOUS_REPLICAS
      value: "{{harbor_psql_replicas}}"
    - name: POSTGRES_CLUSTER_APP_NAME
      value: my_application
    - name: POSTGRES_USER
      value: "stolon"
    - name: POSTGRES_PASSWORD
      valueFrom:
        secretKeyRef:
          name: cambricon-postgresql
          key: postgresql-password
    ports:
    - name: postgresql
      containerPort: 5432
    livenessProbe:
      exec:
        command:
        - sh
        - -c
        - exec pg_isready -U "stolon" -h <IP_ADDRESS> -p 5432
      initialDelaySeconds: 30
      periodSeconds: 10
      timeoutSeconds: 5
      successThreshold: 1
      failureThreshold: 6
    readinessProbe:
      exec:
        command:
        - sh
        - -c
        - |
          pg_isready -U "stolon" -h <IP_ADDRESS> -p 5432
          [ -f /opt/bitnami/postgresql/tmp/.initialized ]
      initialDelaySeconds: 5
      periodSeconds: 10
      timeoutSeconds: 5
      successThreshold: 1
      failureThreshold: 6
    volumeMounts:
    - name: data
      mountPath: /bitnami/postgresql
      subPath:
  volumes:

volumeClaimTemplates:
- metadata:
name: data
spec:
storageClassName: standard
accessModes:
- "ReadWriteOnce"
resources:
requests:
storage: "8Gi"
'''

Hi, @Lebronbingcheng.
Looking at your log:
12:28:02.57 INFO ==> Validating settings in POSTGRESQL_* env vars..
12:28:02.58 INFO ==> Initializing PostgreSQL database...
12:28:02.60 INFO ==> postgresql.conf file not detected. Generating it...
12:28:02.62 INFO ==> pg_hba.conf file not detected. Generating it...
12:28:02.64 INFO ==> Deploying PostgreSQL with persisted data...

It looks like you already have data persisted, so PostgreSQL isn't being configured with the environment variables. Probably your persisted data doesn't have either the repl_user nor the stolon users.
Regards,
Alejandro


[5.7] Remove the last SPI use of _RegexParser symbols (#416)

PatternConverter uses one last AST-based API defined in _StringProcessing, which is technically not allowed but not
problematic b/c PatternConverter also imports _RegexParser. However, this can have compile-time problems later on, so this changes the entry point for PatternConverter to just use an Any parameter that is checked at runtime to actually be AST.
Cherry pick of #416, resolves rdar://92876793.

@swift-ci Please test


Create CODEOWNERS

Thanks!
Note: In gix repos normally put the codeowners file in .github/CODEOWNERS so it's likely to get moved there at some point.  It's not a problem for us, but you might want to be aware of this if you have some programs that assume the location of the file.

@bitdivine All our checks I'm aware of are ok with both locations as they both are documented as acceptable in GitHub. So feel free to move it at any time.

Let's move it, I rather like displaying that file less "up in your face" to devs that fork or browse the repo.
@bitdivine can you accept 👉 https://github.com/dfinity/ic-eth-wallet/pull/253


Migrate from hoa/fastcgi to hollodotme/fast-cgi-client

The goal is to fix #316 and changing the underlying library seems to solve the problem. This PR includes #318.
https://github.com/hollodotme/fast-cgi-client seems to be more maintained and used as well.
This may also solve #274 🤞
ping @hollodotme for information :)

@mnapoli one important metric that we should be considering is performance, here my test:
https://prnt.sc/ngwe5q
The first part is using  hollodotme and the faster one is Hoa.

@atrope thank you for testing, it is indeed something I wanted to test. I have the feeling that hollodotme's library is reconnecting to PHP-FPM every time. I need to dig into this 🤔

@mnapoli Hi, thanks for considering my library for integration!
Correct, the lib currently opens a new connection to php-fpm for each request and disconnects as soon as the response was fetched. TBH, I never considered re-using the sockets for succeeding requests and now I'm asking myself why I didn't. :-/
However I quickly prototyped the re-using of sockets into the lib and that seems to actually work. :) (https://github.com/hollodotme/fast-cgi-client/pull/33)
Looks like I would need some days to finish a new release with proper implementation and tests.

@hollodotme awesome!

@hollodotme sorry for the double ping I just thought about that:
https://github.com/mnapoli/bref/pull/319/files#diff-08c4870f917ac32beb81b794eb61b760L120
When communication via the socket fails we needed to explicitly reconnect with Hoa. I'm guessing that's something that will be necessary as well for you if you make connections persistent, so you may want to expose a public disconnect() or reconnect() method.

@mnapoli There are several checks in the library if the socket is broken. In those cases the Client class (which you use) would/should create a new connection anyway. The lib tries to avoid that the user has to deal with the socket or connections directly.
Is there any specific kind of "socket fails" that I can test against?

Nice!

Is there any specific kind of "socket fails" that I can test against?

What we test in Bref are timeouts: https://github.com/mnapoli/bref/blob/883d74ab410b99b585afd2109fa0dbedec985414/tests/Runtime/PhpFpmTest.php#L724-L746 (the file that is invoked simply does a sleep() until a timeout occurs).
There is also https://github.com/mnapoli/bref/issues/274#issuecomment-487029070 which is a separate issue (we haven't found the root cause), but I'm not sure how it relates to this discussion TBH.

@mnapoli I just released v2.7.0 of the library that now automatically re-uses idle socket connections on successive requests.
In the following cases a new connection will be established:

There is no connected socket
All connected sockets are either busy or not usable anymore, which means

their responses were not ready or received, yet (= busy)
the stream meta data indicates a time out, an EOF or unread bytes (= not usable)
the stream was (externally) closed or terminated (= not usable)

If non of this is true, the first idle connection from the internal collection is re-used for the next request.
In practice that means that all successive synchronous requests will use the same connection as long as the stream is usable.
For successive asynchronous requests it depends on the fact when a socket becomes idle again to be re-used (and how the triggering of requests is organized in call-flow).
I hope this helps to improve performance and reduce resource allocation.

@hollodotme that is awesome! I have updated the required version in this PR. I'll try to do some performance tests (@atrope if you are able to test as well that would be great).

I did some performance tests with https://github.com/brefphp/bref-benchmark and numbers are looking good!
It even seem to drop from 2ms to 1ms for HTTP response time of a echo "Hello world" application. I'm not sure if it is actually significant or not since we are alreay close to very low latencies, but it could be a performance improvement!
Hopefully @atrope and others will be able to test this and confirm that so we can be extra confident.

Hey @mnapoli and @hollodotme I Updated to 2.7 and tested in production for 10 minutes.
It did not solved #274 and the performance was worse than before :-/
https://prnt.sc/ni0mpt
@mnapoli Do we need to enable something in phpFPM file? I Just Updated to 2.7 and used the PR's file (adapted to use ALB).

Does @hollodotme's implementation actually handle multiple headers with the same name like Set-Cookie correctly? Hoa's library does not and multiple cookies can't be returned in a response without overriding the library in a specific spot I added in a recent PR.

@atrope thank you for testing.  I'll try with a real application on my side as well.
@taylorotwell AFAIK no, @atrope was mentioning that as well and if I recall correctly it would be easier to implement with the new library.

@taylorotwell, I fear the answer is no for the current version, as long as you don't want to parse the raw response (which is accessible and includes the header lines) yourself.
I'll see what I can do in the next days. But it'll most likely be a BC break. 🙃

Has the issue been raised with the Hoa/FastCGI maintainer? Perhaps it can be fixed on that end without swapping to an entirely new library?

@taylorotwell With Hollodotme it is doable in fpm file in our side.
If @hollodotme Wants i can easily make a PR to include in his library also.
We just need a flag to enable Multi Header and if it is enabled return each Header as an array.
I Did it in my end:

    /**
     * Return an array of the response headers.
     *
     * @param ProvidesResponseData $response
     * @param boolean $isMultiHeader
     */
    private function getHeaders($response,$isMultiHeader): array
    {
      if ($isMultiHeader){
        $responseHeaders = [];
        $lines  = explode( PHP_EOL, $response->getRawResponse() );
        foreach ($lines as $i => $line ){
          if (preg_match('#^([^\:]+):(.*)$#', $line, $matches )){
            $key = trim( $matches[1]);
            if (!array_key_exists($key, $responseHeaders)) $responseHeaders[$key]= [];
            $responseHeaders[$key][] = trim($matches[2]);
            continue;
          }
          break;
        }
      }
      else {
        $responseHeaders = $response->getHeaders();
      }
      return array_change_key_case($responseHeaders, CASE_LOWER);
    }

@taylorotwell and yes, i 've made a PR there:
https://github.com/hoaproject/Fastcgi/pull/28

In Hollodotme's library just need to edit parseHeadersAndBody and add isMultiHeader variable to constructor, default as false, and a way to change it.
if ( preg_match( self::HEADER_PATTERN, $line, $matches ) )
			{
				$offset                               = $i;
				$name = trim( $matches[1] );
				$value = trim( $matches[2] );
				if ($this->isMultiHeader){
					if (!array_key_exists($key, $this->headers)){
						$this->headers[$key]= [];
					}
					$this->headers[$key][] = $value;
				}
				else {
					$this->headers[ $name ] = $value;
				}
				continue;
			}

I am trying to dig into performances, here is a simple script I'm trying:
require_once __DIR__ . '/vendor/autoload.php';

$fpm = new Bref\Runtime\PhpFpm(__DIR__ . '/tests/Runtime/PhpFpm/request.php', __DIR__ . '/tests/Runtime/PhpFpm/php-fpm.conf');
$fpm->start();

for ($i = 0; $i <= 1000; $i++) {
    $response = $fpm->proxy([
        'httpMethod' => 'GET',
        'path' => '/hello',
        'requestContext' => [
            'protocol' => 'HTTP/1.1',
        ],
    ]);
}

Here is the Blackfire diff between master and this PR:
https://blackfire.io/profiles/compare/529fc635-88eb-4616-902b-75314b334fb2/graph
As you can see this PR runs 40% faster.
One of the explanation is because Hoa seems to be disconnecting/reconnecting every time (which surprises me):

But beyond that Hoa is also slightly slower at reading the response. @atrope can you confirm that your screenshot wasn't all cold starts?
@taylorotwell we could indeed open an issue. TBH this new library seems to be more maintained (https://github.com/hoaproject/Fastcgi/graphs/contributors) and used, Hoa is dragging behind in new releases and the last security issue wasn't fixed upstream (https://github.com/hoaproject/Fastcgi/issues). There is also the Hoa magic around autoloading that makes it not so nice to use. In any case we could stick with Hoa if this issue is fixed upstream, nothing against it. I just don't want to hold my breath if something better is right there ^^ TBH I started this branch not expecting it to work, but since it does I've opened the PR.

Just to be safe I tried with a larger JSON response and got the same results.

@atrope, @taylorotwell  I filed an issue (hollodotme/fast-cgi-client#35) for me to add the multi-headers to the library. It will come in v3.0.0 as it is a BC break. I think I can finish this release until the end of the week.

@atrope I just thought about https://github.com/mnapoli/bref/issues/274#issuecomment-486800345: could it be that response times are worst because of the problem you are facing in #274? I.e. responses are broken, there is a lot of socket reconnect going on, etc.

@mnapoli Yes i can assure it was not 10 minutes of Cold Starts :)
And while i cannot confirm or deny that the worst performance is caused by #274 , i think the evidence indicates that it's not that, because the only thing that changes from both performance tests is the underlying library.
So Hoa's is also facing that problem but performance is still twice better than Hollodotme (I do prefer Hollodotme library, as i stated, that's the one i used before, and as i also stated before, When migrated to bref it came with a performance benefit that we are know finding out was the library).
Checking your tests I Still think that it could be a simple configuration and i will dig further today.
DO You know any way i can debug on prod so we can see where the latency is coming from?

DO You know any way i can debug on prod so we can see where the latency is coming from?

Not really. You could edit the PhpFpm class and log custom CloudWatch metrics, it would be a little bit of effort though.
Or you could also try to run a benchmark locally on your PC, without SAM (run it with PHP-FPM) and use https://blackfire.io/ to measure the performance differences. You can use blackfire run php instead of php and that's it. For my tests above I ran blackfire run php perf.php.

I just released v2.7.1 with a bugfix related to #274.

@atrope I can create a debug layer/function for you. What tools or functionality would you like in it?

Is this performance issue a blocker for this PR? (Just wondering if it will be merged soon)

@gonzalovilaseca I would really like to figure out that performance problem before merging :/

Ok, np. If I can help just let me know what I can do :-)

@bubba-h57 anything that would let me see where is the added latency because the code is the same only bref's phpfpm file changes.

If you are able to run your app locally you could profile the difference (xdebug/blackfire/etc.). But if the difference is because of something that happens on Lambda (or at scale) then that won't help ^^

Ok now that my Environment is fixed we can see a better performance from hollodotme.
WHat was happening is the contrary, because lots of requests were failing the lambda time were less than 50ms making the average looks better.
Now when i changed from Hoa to hollodotme i saw a 10ms increase but i think it would be wise for us to make a one-day test for each library. i will test Hoa tomorrow without the errors, and the day after tomorrow i will put hollodotme 2.7.1 for one whole day also, that way we will know for sure if we have or not performance problems.
What do you say @mnapoli  are you in a hurry to merge this PR or i can make this Test( I Will do anyway)?

OK that's good to know!
Take your time to test tomorrow and let us know. If all goes well I can try to do a release wednesday or thursday.

I released 3.0.0-alpha of the library containing the new method and behavior for multi-value headers and the removal of the previously deprecated Response#getRawResponse() method in favor of Response#getOutput().

BC breaks are documented here
New header behavior is documented here

I still want to add some more tests and maybe one or two methods to the request classes for more user convenience. That's why it's an alpha "only".

Hey :-), if the issue comes from hoa/fastcgi, it's easily fixable. The size of the HTTP response is hardcoded for obvious reason, but we can increase it easily. Do you need that fix?

Also, about connection and disconnection, if you can provide me a script to replicate, I can fix that quite easily too.

@mnapoli  Ok, The performance was pretty much the same between HOA and HOLLODOTME.
:+1:

@atrope @mnapoli Do you need hoa/fastcgi to support connecting/disconnecting once, and not per send call? It's easy to integrate.

If you want to use the xdebug profiler in Lambda:
arn:aws:lambda:us-east-1:965741605173:layer:bref-php-73-xdebug:1 add that layer and activate the extension at /opt/sts/modules/xdebug.so
I only have a US-EAST-1 layer published.

OK thanks everyone for testing, @Hywan thanks for the heads up.
I think we should move ahead with this change, hopefully that's ok with everyone 🤞 I've been thinking about this these last few days:

the work here is done already and the bug is fixed, it's ready to merge
for my own perspective, the hollodotme library is definitely easier for me to dig into in case I need to debug or even maintain (if that's ever necessary), the Hoa code is split over many repositories and uses patterns and PHP practices I'm not familiar with (I'm thinking for example about the custom autoloading)
better DX with the hollodotme library as a maintainer (that's purely from my personal POV), in Hoa classes are magically resolved and this breaks several features of the IDE
so far it seems that the hollodotme library has more active users (issues vs issues) and shorter maintenance cycles (stats vs stats to illustrate)
no Composer dependencies in the hollodotme library (a really negligible advantage I agree)

no Composer dependencies in the hollodotme library <- this is the most awesome advantage evah!


[Documentation] minor typo in CONTRIBUTING.md

very minor typo, so didn't create ticket.

Hi,
I don't think it's a typo. You should indeed check that the issue doesn't already exist before creating a new one.

Right, my fault, close it.


Question about inference/cloud/translation/gnmt/tensorflow/

I'm using tensorflow-gpu version ot Turing T4 for inferencing benchmarking.
Is the benchmark really using GPU for inferencing?
I got the code from mlperf/inference/cloud/translation/gnmt/tensorflow/.
I increased the # of iterations for 1 to 500000 in run_task.py.
It seems that the GPU is not being used for inferencing. Initially for first few seconds GPU utilization is 30%.  Then nvidia-smi reports GPU utilization to be zero for most of the time for 500000 iterations.
It takes about 525 seconds to complete the run. For the first 125 or seconds, nvidia-smi shows any GPU utilization. After that nvidia-smi shows 0 utilization. Is the benchmark really using GPU for inferencing?
Here is is excerpt form the log of the run:
dynamic_seq2seq/decoder/output_projection/kernel:0, (1024, 32316),
2019-03-08 14:08:50.113076: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1484] Adding visible gpu devices: 0
2019-03-08 14:08:50.113130: I tensorflow/core/common_runtime/gpu/gpu_device.cc:965] Device interconnect StreamExecutor with strength 1 edge matrix:
2019-03-08 14:08:50.113139: I tensorflow/core/common_runtime/gpu/gpu_device.cc:971]      0
2019-03-08 14:08:50.113145: I tensorflow/core/common_runtime/gpu/gpu_device.cc:984] 0:   N
2019-03-08 14:08:50.113335: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1097] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 3318 MB memory) -> physical GPU (device: 0, name: GRID T4-4Q, pci bus id: 0000:02:01.0, compute capability: 7.5)
loaded infer model parameters from /home/ml/inference-master/cloud/translation/gnmt/tensorflow/ende_gnmt_model_4_layer/translate.ckpt, time 1.01s
Start decoding
decoding to output /home/ml/inference-master/cloud/translation/gnmt/tensorflow/nmt/data/output/g_nmt-out
Iterations  500000
done, num sentences 3003, num translations per input 1, time 35s, Fri Mar  8 14:09:27 2019.
done, num sentences 3003, num translations per input 1, time 35s, Fri Mar  8 14:09:27 2019.
done, num sentences 3003, num translations per input 1, time 35s, Fri Mar  8 14:09:27 2019.
done, num sentences 3003, num translations per input 1, time 35s, Fri Mar  8 14:09:27 2019.
done, num sentences 3003, num translations per input 1, time 35s, Fri Mar  8 14:09:27 2019.
done, num sentences 3003, num translations per input 1, time 35s, Fri Mar  8 14:09:27 2019.
Excerpt from nvidia-smi log: GPU utilization drops from 26% to 0 and then stays 0 for the rest of the run.
Fri Mar  8 14:09:27 2019
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 410.92       Driver Version: 410.92       CUDA Version: 10.0     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  GRID T4-4Q          On   | 00000000:02:01.0 Off |                  N/A |
| N/A   N/A    P0    N/A /  N/A |   3851MiB /  4096MiB |     26%      Default |
+-------------------------------+----------------------+----------------------+
| N/A   N/A    P0    N/A /  N/A |   3851MiB /  4096MiB |     26%      Default |
| N/A   N/A    P0    N/A /  N/A |   3851MiB /  4096MiB |     26%      Default |
| N/A   N/A    P0    N/A /  N/A |   3851MiB /  4096MiB |     25%      Default |
| N/A   N/A    P0    N/A /  N/A |   3851MiB /  4096MiB |     24%      Default |
| N/A   N/A    P0    N/A /  N/A |   3851MiB /  4096MiB |     24%      Default |
| N/A   N/A    P0    N/A /  N/A |   3851MiB /  4096MiB |     23%      Default |
| N/A   N/A    P0    N/A /  N/A |   3851MiB /  4096MiB |     22%      Default |
| N/A   N/A    P0    N/A /  N/A |   3851MiB /  4096MiB |     21%      Default |
| N/A   N/A    P0    N/A /  N/A |   3851MiB /  4096MiB |     21%      Default |
| N/A   N/A    P0    N/A /  N/A |   3851MiB /  4096MiB |     20%      Default |
| N/A   N/A    P0    N/A /  N/A |   3851MiB /  4096MiB |     19%      Default |
| N/A   N/A    P0    N/A /  N/A |   3851MiB /  4096MiB |     18%      Default |
| N/A   N/A    P0    N/A /  N/A |   3851MiB /  4096MiB |     18%      Default |
| N/A   N/A    P0    N/A /  N/A |   3851MiB /  4096MiB |     17%      Default |
| N/A   N/A    P0    N/A /  N/A |   3851MiB /  4096MiB |     16%      Default |
| N/A   N/A    P0    N/A /  N/A |   3851MiB /  4096MiB |     15%      Default |
| N/A   N/A    P0    N/A /  N/A |   3851MiB /  4096MiB |     14%      Default |
| N/A   N/A    P0    N/A /  N/A |   3851MiB /  4096MiB |     14%      Default |
| N/A   N/A    P0    N/A /  N/A |   3851MiB /  4096MiB |     13%      Default |
| N/A   N/A    P0    N/A /  N/A |   3851MiB /  4096MiB |     12%      Default |
| N/A   N/A    P0    N/A /  N/A |   3851MiB /  4096MiB |     11%      Default |
| N/A   N/A    P0    N/A /  N/A |   3851MiB /  4096MiB |     11%      Default |
| N/A   N/A    P0    N/A /  N/A |   3851MiB /  4096MiB |     10%      Default |
| N/A   N/A    P0    N/A /  N/A |   3851MiB /  4096MiB |      9%      Default |
| N/A   N/A    P0    N/A /  N/A |   3851MiB /  4096MiB |      8%      Default |
| N/A   N/A    P0    N/A /  N/A |   3851MiB /  4096MiB |      8%      Default |
| N/A   N/A    P0    N/A /  N/A |   3851MiB /  4096MiB |      7%      Default |
| N/A   N/A    P0    N/A /  N/A |   3851MiB /  4096MiB |      6%      Default |
| N/A   N/A    P0    N/A /  N/A |   3851MiB /  4096MiB |      5%      Default |
| N/A   N/A    P0    N/A /  N/A |   3851MiB /  4096MiB |      5%      Default |
| N/A   N/A    P0    N/A /  N/A |   3851MiB /  4096MiB |      4%      Default |
| N/A   N/A    P0    N/A /  N/A |   3851MiB /  4096MiB |      3%      Default |
| N/A   N/A    P0    N/A /  N/A |   3851MiB /  4096MiB |      2%      Default |
| N/A   N/A    P0    N/A /  N/A |   3851MiB /  4096MiB |      2%      Default |
| N/A   N/A    P0    N/A /  N/A |   3851MiB /  4096MiB |      1%      Default |
| N/A   N/A    P0    N/A /  N/A |   3851MiB /  4096MiB |      0%      Default |
| N/A   N/A    P0    N/A /  N/A |   3851MiB /  4096MiB |      0%      Default |
| N/A   N/A    P0    N/A /  N/A |   3851MiB /  4096MiB |      0%      Default |
| N/A   N/A    P0    N/A /  N/A |   3851MiB /  4096MiB |      0%      Default |
| N/A   N/A    P0    N/A /  N/A |   3851MiB /  4096MiB |      0%      Default |
| N/A   N/A    P0    N/A /  N/A |   3851MiB /  4096MiB |      0%      Default |
| N/A   N/A    P0    N/A /  N/A |   3851MiB /  4096MiB |      0%      Default |
| N/A   N/A    P0    N/A /  N/A |   3851MiB /  4096MiB |      0%      Default |
| N/A   N/A    P0    N/A /  N/A |   3851MiB /  4096MiB |      0%      Default |
| N/A   N/A    P0    N/A /  N/A |   3851MiB /  4096MiB |      0%      Default |
Thanks,
-Uday

@ukurkure I'd like to wait for the MLPerf load generators to integrated then we will see if this issue still exist. I expect the code to change dramatically with the load generator.

Hi @christ1ne when is expected the release of the MLPerf load generator?

please test the latest GNMT code with the LoadGen & re-open the issue if needed. Thanks @ukurkure


MacOS fix for md5sum, and matched password to README.md

Actually a fix for any system missing md5sum, runs it in a container instead.

I want @mriccell to review the password change as well just so she's aware.

@ericsmalling could you please sign (or point me towards your existing) Oracle Contributor License Agreement?


Only resolve paths and skip loading

I'd like to use cosmiconfig for it's advanced finding/resolving capabilities. I do not require file loading or parsing.
Is there an easy way to have cosmiconfig resolve a file path and skip loading?

Nevermind I just found #176


add automate exception message for invalid enum value as api request parameter

Describe your changes:
Fixes

When enum value is incorrect in api call, the invalidEnumValue()method generates the exactly message as we expect. for example the Include.java in databaseServiceResource.java list() api. There is a unit test example that shows how to use invalidEnumValue() method instead of traditional way i.e. writing plain string.

Type of change:

[ ] Bug fix
[x] Improvement
[ ] New feature
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] Documentation

Checklist:

[x] I have read the CONTRIBUTING document.
[ ] My PR title is Fixes <issue-number>: <short explanation>
[ ] I have commented on my code, particularly in hard-to-understand areas.
[ ] For JSON Schema changes: I updated the migration scripts or explained why it is not needed.

@EsmaeilChitgar left comments on the PR. You have build failures in the PR that's causing tests to fail.
You can also run the tests locally
mvn clean package

@EsmaeilChitgar left comments, can you please take a look

Thanks @EsmaeilChitgar will merge in once all the checks passes


691 - Allow external schema refs

@kylehodgetts as per our slack convo
The operator should not be able to get external reference from URLs or from relative paths. It simply doesn't have that ability and it should remain so.
In order to solve this issue let's go with the client side i.e. kusk deploy and kusk api generate will be able to "follow" references and add it as actual values in the API manifest.


Springless Jira REST client

It'd be nice if we could exclude Spring from the Jira extension. But it's currently not really possible because some of the Jira REST client internals depend on the Spring DisposableBean.
I opened an issue to see if Spring could be made non-mandatory.
https://ecosystem.atlassian.net/browse/JRJC-258
If that turns out not to be possible, then we could consider implementing a custom JiraRestClientFactory & JiraRestClient based on Vert.x without the requirement of any Spring bits.

Given there's been no movement on fixing JRJC-258, one possible solution to this, would be to have some build time magic that would result in not requiring Spring.
Either:
Generate the DisposableBean interface (it only has one method)
Or
Modify ApacheAsyncHttpClient to implement interface com.atlassian.jira.rest.client.internal.async.DisposableHttpClient instead of DisposableBean. They both have identical destroy method signatures.


Training error in multibox_loss.py

I am very new about Pytorch, and when I use the default setting, and train the voc firstly, get the following error:

Loading base network...
Initializing weights...
Loading Dataset...
Training SSD on VOC0712
Traceback (most recent call last):
File "train.py", line 231, in
train()
File "train.py", line 183, in train
loss_l, loss_c = criterion(out, targets)
File "/home/mengyong/anaconda3/lib/python3.6/site-packages/torch/nn/modules/module.py", line 325, in call
result = self.forward(*input, **kwargs)
File "/home/mengyong/Detection/ssd.pytorch/layers/modules/multibox_loss.py", line 83, in forward
num_pos = pos.sum(keepdim=True)
RuntimeError: sum() missing 1 required positional arguments: "dim"

Can you help me solve the problem, very thanks!

I met the same problem, have you solved it?

Yes, just set the error line
num_pos = pos.sum(keepdim=True)
as the new coding
num_pos = pos.sum(dim=1, keepdim=True)
all thing will be OK!
Importantly, you will find num_pos is  defined newly in the following line, so you can just remove it, things also will be OK

@lmingyin Thanks!

@lmingyin hello,,i have some training issues?


[Suggestion] Add Charged Creepers as a spawnable mob

This adds #1.
Feature

Adds charged creepers as a spawnable mob

The charged creeper has a 30% chance of spawning in place of a normal creeper.
Shooting it will discharge it, turning it into a normal creeper.
The charged creeper is more powerful and deals more damage than a normal creeper.

Added that feature, along with a way to configure charged creeper chances
Breaking Change:
Modifies standard.json to add:

"charged_spawn_stage": the stage to start spawning charged creepers (default: 3)
"charged_spawn_percent": the chance for a creeper to be charged (default: 35)


CocoaPods support

Firebase is cocoapods only, so cocoapods support for this library kinda makes sense :)

would you mind checking on this? I am showing this in pod install..
[!] Unable to find a specification for CodableFirebase``
Happy new year!


unable to display custom even slot

Hi,
nice job on the calendar and the documentation. I have set up everything I needed in no time! There is just one thing that I just cant figure out how to do - I can't figure out how to render my own html in inside an event (and access the event's object properties). It seems to me it is possible from the documentation, but I guess I am doing something wrong...
I am using the "3.0.0-alpha.4".
<vue-cal default-view="day"
         id="vuecal"
         @ready="scrollToEarliestTimeslot"
         :selected-date="selectedDate"
         :events="calendarEvents"
         :timeCellHeight="timeCellHeight"
         :time-step="minTimeslotLength"
         hide-title-bar
         :split-days="calendarResourceObjects"
         :on-event-click="onTimeslotSelect">
  <template v-slot:event="{ event, view }">
    zaza
    {{ event }}
  </template>
</vue-cal>

Neither 'zaza' or event are rendered in the day view (or any other view), and the event or view variables are not available inside the template component. It just renders the events as if the template wasnt even there.

I have tried v-slot:event and v-slot:event-renderer, but no luck.

aaaaand I figured it out thanks to writing this issue :). I was using an old version of vue, so the scope syntax didn't work. after upgrading it works perfectly...

Hi @peachykeeen,
Sorry I missed your question.
Yes at the moment the documentation is up to date with the alpha version which should be released as stable soon. So this is temporary and should not last too long.
Anyway, I'm glad you found out it was a Vue version mismatch.


leptos 0.7  compatible?

is this compatible with leptos 0.7?

Not yet, but I'm on it migrating to 0.7.


Avoid pinning exact versions of dependencies

Currently all of the pypi dependencies are pinned to exact versions ("=="). Typically dependencies are specified with min versions (">=") to avoid any conflict with other packages in the environment.

/assign @karthikv2k


Improve the structure of job_documents

Motivation
Currently, it is not well documented what job_documents store and how they are structured. This might become an issue for further development. The intention of the job_document.json is to store everything that is related to state of a simulation and is not required to identify a given job as unique (that would be in the signac_statepoint.json), furthermore the job document is used for queries of results.
Structure
Several categories for data in the job document can be identified:

Category
Description
Key
Type

results
Data that is produced by a simulation, should consider multiple runs
results
list[dict]

cache
Things that are stored to avoid expensive reevaluation which should be constant for this job, eg number of cells
cache
dict

history
List of operations that have been applied to a simulation and the corresponding result
history
list[dict]

state
all state relevant properties
state
dict

Currently, we have the following key value pairs in job documents:

operation_hash:  used to check if job doc needs to be setup (needed?)
base_id: parent of job (rename to parent_id and move to statepoint?)
keys: keys of this specific job (move to state_point)
pre_build: list of pre build operations
post_build: list of post build operations
state: state of the job ready|success|failure|started|tmp_lock
init_dependent
obr: stores further data like solver and logfiles

Archival
Additionally, things are getting more complicated for archival of results into a data repository. Here we have data from different sources and currently request that the job documents can be recreated just from the log files to avoid any potential merge conflicts. This however destroys any history and intermediate results. To avoid any merge conflicts, archival-mode should store job_documents as separate file on archival eg. archive_document_<cluster_name>.json from which the job_documents can be recreated in a separate step.

@greole  this is resolved by #121, right?

Thanks!


CSS Save button border styling

Save button border does not show up evenly, needs to be fixed appropriately.

Borders were fixed so that they are even


[Future][Not sure] Run Sync in the SW

Currently the app only works in one tab. If the user tries to use the same profile in multiple browser tabs, there will be a sync conflict and the workspace will be broken.
To solve this, one idea is to move Sync related things to the service worker. This also makes the Sync running in the background when the application is not running.

Presently, NDNts is not verified for running in service workers.
I'm willing to implement such support, as long as the main requirements are working:

Web Crypto API
IndexedDB
HTTP/3 WebTransport
WebSocket


Pass generated code version to http headers

resolves #1077

Codecov Report

Merging #1079 into master will decrease coverage by -0.26%.
The diff coverage is 100%.

@@             Coverage Diff              @@
##             master    #1079      +/-   ##
============================================
- Coverage     85.18%   84.93%   -0.26%
+ Complexity     3488     3478      -10
============================================
  Files           367      367
  Lines         13868    13869       +1
  Branches       1579     1579
============================================
- Hits          11814    11780      -34
- Misses         1598     1634      +36
+ Partials        456      455       -1

Impacted Files
Coverage Δ
Complexity Δ

.../com/google/api/codegen/ruby/RubyGapicContext.java
75.62% <100%> (+0.24%)
42 <1> (+1)
:white_check_mark:

...le/api/codegen/gapic/MainGapicProviderFactory.java
94.21% <100%> (-0.02%)
16 <0> (ø)

...gen/transformer/ruby/RubyPackageMetadataNamer.java
0% <0%> (-91.67%)
0% <0%> (-4%)

...ansformer/ruby/RubyPackageMetadataTransformer.java
20% <0%> (-80%)
2% <0%> (-7%)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9fa58ac...acd3169. Read the comment docs.

SGTM!


Crash when notebook scrolling is on and a lot of output is streamed in

Does this issue occur when all extensions are disabled?: Yes/No

VS Code Version:
OS Version:
Version: 1.79.0-insider (user setup)
Commit: 2575777b9d3cdb865d61f9f1dfaac1fea9e1bde6
Date: 2023-05-12T05:21:06.392Z
Electron: 22.5.2
Chromium: 108.0.5359.215
Node.js: 16.17.1
V8: <IP_ADDRESS>-electron.0
OS: Windows_NT x64 10.0.19045
Sandboxed: Yes

Steps to Reproduce:

Install Jupyter and Python
Ensure user setting "notebook.output.scrolling": true, "notebook.output.textLineLimit": 30 (this is the default)
Create interactive window
Run cell containing

import sys
for i in range(31500):
    print('{!r} {!r}'.format(list(range(10)), i), file=sys.stderr)

This works, it's only abou 1 MB of output.

Run cell containing

import sys
for i in range(31500):
    print('{!r} {!r}'.format(list(range(10)), i), file=sys.stderr)
    sys.stderr.flush()

The renderer crashes!

@rebornix @DonJayamanne are you accepting PRs for this?
It looks like, there is no batching of events from the notebook, although you made processing each event more efficient last year.
The renderer is calculating the scroll height each time stderr is flushed

I guess OutputCell.updateContentAndRender needs to be debounced with requestAnimationFrame, but, when I tried it the resize observer started intermittently sending height of 0 back to the extension.

@r3m0t thanks for offering help, it would be very great. We might need to carefully design the debouncing to ensure a smooth incremental update experience though.

I had a look last week, but I found many concerns. It's not just the renderer. Basically, updateContentAndRender is an idempotent operation (an upsert) of the output into the rendered DOM. The approach of sending the full output to the renderer each time is quadratic-
import sys, json
res = []
total_message_length = 0

for i in range(31500):
    res.append('{!r} {!r}\n'.format(list(range(10)), i))
    message = json.dumps({'event': 'update_output', 'output_id': '01010239032-3210-321-312321321', 'contents': ''.join(res)})
    total_message_length += len(message)
print(f'Sending {total_message_length:,.0f} bytes with a final output of {len(message)}')
# Sending 18,556,686,495 bytes with a final output of 1185979

That's really excessive, and there's a lot of work needed in the output renderer as well. Each time it processes all the last 5000 lines for ANSI colors and links, instead of just the new part, and creates new div elements - https://github.com/microsoft/vscode/blob/143d9674dd470c8f3d3a5e4b930f5eb1fe9a91d3/extensions/notebook-renderers/src/textHelper.ts#L94-L104
it would make more sense to send incremental content -
https://github.com/microsoft/vscode/blob/143d9674dd470c8f3d3a5e4b930f5eb1fe9a91d3/src/vs/workbench/contrib/notebook/browser/view/renderers/backLayerWebView.ts#L1492
Separate issue as it's on the main host side- there is also quadratic behaviour in formatStreamText - this is called 31,500 times and searches the full string each time with these two includes calls -
https://github.com/microsoft/vscode/blob/a54803afc9476aff9ada1bb7a44fc05d2edabd34/src/vs/workbench/contrib/notebook/common/notebookCommon.ts#L1071
Basically, debouncing would require updateContentAndRender to stay idempotent which I don't agree with, and it would make the logic more complicated if you want to introduce a mix of idempotent and incremental messages later.
I might be able to send you some small changes to optimize https://github.com/microsoft/vscode/blob/143d9674dd470c8f3d3a5e4b930f5eb1fe9a91d3/extensions/notebook-renderers/src/index.ts , but I don't think performance will be good until you can tackle the above.
Or should I just tell my users to go to Terminal 🤣

Oh, and I think I'm basically suggesting unpicking https://github.com/microsoft/vscode/issues/161023 - or optimizing for both cases (append usually but replace when \r or \b used as stream characters come into play)

We were considering incremental updates to the output here, but decided for a simpler change to fix that issue. As you noted, we look for characters that edit previous outputs and compress the buffer into a single output, which would be very difficult to support if we only sent text to append. We also need to consider 3rd party renderers that would break if we changed the message in such a way.
We have a hard limit of 5000 lines for text output in the renderer, so it might make sense to also limit the message sent to help mitigate this issue.

I don't think third party renderers need to be affected as the escape sequences are already only handled for a few hard coded MIME types. Are there other renderers for these types? Are there other MIME types that need to be handled incrementally? In the Jupyter protocol, you can't incrementally update an output of a MIME type.  This can be private API to the built in stream renderers.
I do agree OutputItem[] is the wrong type when the escape sequence can "affect data behind it". I was thinking of an OutputItem.replaceVersionId which increments  exactly when the new contents is different from the old contents in the range [0, oldContents.length). If it's unchanged then linkify, ANSI color processing and <span> creation can be skipped for the existing output.

Thanks for putting this on the iteration plan, can you tag it freeze-crash?
I'll leave you with my thoughts which may be useful in implementation, hopefully this doesn't send you down the wrong way!

I'm not sure my 18,556,686,495 bytes was accurate as I didn't check the behaviour of copyBufferIfNeeded in this case.
The first question if making big changes to this area is whether to keep stdout and stderr separate or mix them- more like a terminal or Visual Studio 2022 Python support. In VS the stdout and stderr are shown in the same place with different colours. It makes more sense to me and users at my employer than what Jupyter does. Jupyter wire protocol doesn't say how stdout/stderr should be displayed
As I mentioned there's inefficiencies at almost every layer of the stack wrt output.

The efficient one is the ext host and messages to the main process due to the inclusion of ICellExecuteOutputEditDto.append.

But when we reach ExtHostCell.setOutputItems it gets quadratic behaviour and we forget it's an append event.

Then backLayerWebview.ts and a couple of stack frames in the webview side are paying the quadratic price.

As well as the application/vnd.code.notebook.stdout renderer.

I would split CellExecutionUpdateType.OutputItems into two update types - one is always append and only contains application/vnd.code.notebook.stdout and/or stderr and the other is the normal one (which no longer needs an append case).
This follows all down the stack, if \r or \b is encountered then the event gets converted to the normal type of event before it gets to backLayerWebview & etc.
The debouncing is probably worthwhile even with these optimisations, as I couldn't find a generic way to avoid that Layout 10ms cost shown in the profiler, even if it was fully incremental all the way down. Once webviewPreloads.ts knows which showOutput messages are appendy, it'll be easy to implement the debouncing there.
A non-generic way I just thought of is to not measure the offsetHeight of the output element after rendering ( https://github.com/microsoft/vscode/blob/70dff6df08d008e895e3f23013c3d54088d90d50/src/vs/workbench/contrib/notebook/browser/view/renderers/webviewPreloads.ts#L2563 ) . After all, if there are enough lines for the scrollbar to appear, you know the output height will stay constant as line count increases. But, IMO you will hit inefficiencies elsewhere.

PS While writing I noticed MirrorNotebookDocument ignores NotebookOutputItemChangedEvent.append / NotebookOutputChangedEvent.append, I don't know the impact and there is a comment that might mean you already know this but I didn't understand it so I thought it was worth mentioning.

@amunger and I discussed how we can introduce incremental rendering to the stream output renderer and his experiments are looking promising. Here are some perf logs we captured
With incremental rendering

Every output append takes around 2ms and it's very consistent. Our hypothesis is once we reach the max height of the scrollable element, the height will not change anymore and it won't trigger any resize to its ancestor elements. Every append operation is only changing its internal content height, that's why the layout is very minimal and almost identical each time. With https://developer.mozilla.org/en-US/docs/Web/CSS/contain, it can probably be reduced further.
Without incremental rendering

Without incremental update, every update is a "remove" and then "insert" and the "insert" operation is getting bigger, thus the rendering time for each operation is getting longer. Starting from around 2ms, all the way to tens or hundreds of ms


Adding SSL troubleshooting

otherwise +1

this also feels like it's in the wrong place.

@esethna I'm going to commit this and leave the PM Review label on. I agree with Corey this isn't the best place for this troubleshooting guide and will defer to you on where to put it finally, this is just needed to answer some customer issues right now.


Corrected URL for Cornac

Ideally should point to http://www.cornac.info/ but URL is brokem ATM.

Thank's


Fix data channel removal

After closing multiple Terminal7 tabs, webexec sends output multiple time to same data channel. Here's a screenshot after I typed 1-9 & 0.

Here's the log:
2020-11-11T18:14:20.964+0200	INFO	Sending output to 12 dcs
2020-11-11T18:14:20.964+0200	INFO	closing & removing dc because state: "closed"
2020-11-11T18:14:20.964+0200	INFO	closing & removing dc because state: "closed"
2020-11-11T18:14:20.964+0200	INFO	closing & removing dc because state: "closed"
2020-11-11T18:14:20.964+0200	INFO	closing & removing dc because state: "closed"
2020-11-11T18:14:20.964+0200	INFO	closing & removing dc because state: "closed"
2020-11-11T18:14:20.964+0200	INFO	closing & removing dc because state: "closed"
2020-11-11T18:14:20.964+0200	INFO	Sending output to 11 dcs
2020-11-11T18:14:20.964+0200	INFO	closing & removing dc because state: "closed"
2020-11-11T18:14:20.964+0200	INFO	closing & removing dc because state: "closed"
2020-11-11T18:14:20.964+0200	INFO	closing & removing dc because state: "closed"
2020-11-11T18:14:21.147+0200	INFO	Sending output to 10 dcs
2020-11-11T18:14:21.147+0200	INFO	closing & removing dc because state: "closed"
2020-11-11T18:14:21.147+0200	INFO	Sending output to 9 dcs
2020-11-11T18:14:21.147+0200	INFO	closing & removing dc because state: "closed"
2020-11-11T18:14:21.150+0200	INFO	Sending output to 8 dcs
2020-11-11T18:14:21.151+0200	INFO	Sending output to 8 dcs
2020-11-11T18:14:21.286+0200	INFO	Sending output to 8 dcs

I counted 11 "closing & removing" and expected the 12 dcs to go down to 1.

I'm not a 100% sure but IMO this is due to the fact that pane := Panes[id-1] is returing a copy of the pane and the way slices work in Go (append might reallocate).
I'd use the below code to handles panes:
package main

import (
	"fmt"
	"sync"
	"sync/atomic"
)

type PanesDB struct {
	panes map[int]*Pane
	m     sync.RWMutex
	id    int64
}

func NewPanesDB() *PanesDB {
	return &PanesDB{panes: make(map[int]*Pane)}
}

func (pd *PanesDB) Add(p *Pane) {
	pd.m.Lock()
	defer pd.m.Unlock()

	p.ID = pd.newID()
	pd.panes[p.ID] = p
}

func (pd *PanesDB) Delete(id int) error {
	pd.m.Lock()
	defer pd.m.Unlock()

	_, ok := pd.panes[id]
	if !ok {
		return fmt.Errorf("pane %#v not found", id)
	}
	delete(pd.panes, id)
	return nil
}

func (pd *PanesDB) Get(id int) *Pane {
	pd.m.RLock()
	defer pd.m.RUnlock()

	return pd.panes[id]
}

func (pd *PanesDB) newID() int {
	return int(atomic.AddInt64(&pd.id, 1))
}

@daonb See c3241d1, IMO it fixes the problem but now the TestPayloadOperations segfaults.
LMK if you want me to look further.

I like what you did with the panes, but it's not enough. Every pane has a slice of dcs - one for each connected client - and that's where our problem is. We also need a dcs_db.go so that each pane can have a db of data channels and we can easily remove a datachannel. I did it, updated the branch, merged to master and closed the bug.


Pull in external dimension definitions, and map to values found in dataset

When a user states that a variable should be mapped to a known dimension, the system should offer a way to pull in the definition of that dimension, and (if needed) map possible values from its code list to the values found in the dataset.
This can be achieved in three ways:

Check whether we already know the dimension in our triple store (ASK), then query it directly (SELECT)
Use a follow-your-nose approach by resolving URIs as needed until we have all information needed (e.g. SDMX is defined across multiple files)
Query a SPARQL endpoint for the data (e.g. http://lod.openlinksw.com/sparql)

We now do all three. But the mapping to the values in our dataset has not been implemented yet (pull down menus are shown alongside values, but there is no mapping in the backend yet)


Added support for binary releases

Added support for publishing binaries on release page

Work was continued on my fork for now.


Update Makefile and switch to GH Actions

Align the Makefile with what we do in CSI, and update from Travis to GH Actions

Nah, the errors unfortunately were not trivial. Bogged me down for a good few hours. I will get CI clean, and then update it here. Close....


change nav so that it's dropdown

if possible, change the navigation so that it's a dropdown menu: opening one day, it opens all the novellas within that day

Some information on this:
Collections are a jekyll feature, not a minimal-mistakes feature. More information here: https://jekyllrb.com/docs/collections/
Also - a collection is rendered as a
 with each item within it being an
. When you add a title:  tag in the sidebar menu list, that puts everything until the next title:  into an
 inside the main list item.

          Parent Page A

Child Page A1

Child Page A2

Child Page A3

          Parent Page B
and so on

this means that you should be able to identify the correct grouping to show/hide using the following CSS selector:
ul.nav_items > li > li
javascript for showing/hiding a chunk of text:
https://www.w3schools.com/howto/howto_js_collapsible.asp
If the default will be a display: none for the closed section, how can we make it more robust, in case there is no js, or the js breaks?


Error when file create on remote

File created in folder on remote, trsync fail with:
[2021-10-09T19:39:11Z INFO  trsync::remote] remote event : content.created.file (2988)
[2021-10-09T19:39:11Z INFO  trsync::operation] Operation : NewRemoteFile(2988)
[2021-10-09T19:39:11Z ERROR trsync::operation] UnexpectedError("RequestError(\"Error happen when make request : reqwest::Error { kind: Decode, source: Error(\\\"missing field `content_id`\\\", line: 1, column: 100) }\")")

done


Explore softer white / black palette

Just condensed the color palette to true monochrome:
The whites and the darks are part of the same hue.

This miiiight exacerbate the visual "flow" point you brought up, @gareththegeek , but I'll take a crack at that next.

Darkened the #2 color quite substantially - #004C48

Let me know if you think it should still be darker.

I think it's a good balance between the black while still showing enough color saturation.


PixiHTMLContainer has his own PointerContext-Coordinates

Actions:

Add a PointerContext to the canvas
Add a PixiHTMLContainer to the canvas

Expected behaviour
The PointerContext holds the coordinates of the cursor inside the canvas (see image).

It should hold this coordinates over all elements inside the canvas.
Found behaviour
The PointerContext coordinates set a new coordinate system if hover over a PixiHTMLContainer (see image).

And please tell me: is it okey that I open so much issues? I don't want you to be uncomfortable. At the time I start with your library I don't expect the role of a beta tester. But the more I do with it, the more likely I'll stick with it. And my project will probably drag on for a while.
So if you don't want me as your beta tester, say so now. Since the various react-pixi-libraries are incompatible (at least I didn't see anything of the sort) I will depend on your bug fixes.
I would like this role :D I'm learning a lot from it.

My best guess on this one is that by hovering over a different HTML element, the event intercepted by the context is not the one dispatched by the canvas. Actually I think that the fact that they use the same context may be the cause of the issue.
I need to investigate this one

New version is 1.3.24

Now it works! Thank you.


When authenticating requests to the Twitter API v2 endpoints, you must use keys and tokens from a Twitter developer App that is attached to a Project.

Describe the bug
This error shows when using a new created dev account with either the default project and app or new project and app with the free plan.

+1 Same issue

Same issue. New account, app is attached to a project.

same here even app is attached to a project. No issue on create tweet endpoint

Same! new account/project same issue :[

Same！Has anyone tested paid use?

Same issue here

If you are using an old twitter api before projects existed you need to create a project and associate it with the existing old app before it will work under the new settings.

If you are using an old twitter api before projects existed you need to create a project and associate it with the existing old app before it will work under the new settings.

Could you repeat that?
Not sure I understand what you mean.
I had an app within a project using V2 which was working fine until the whole plan change thing.
I now have the basic $100 access, but still get the same error when I access the search tweets endpoint.

Same issue here.

It appears twitter has locked this behind Pro. I have tested Basic and it does not work. Can someone confirm if this works with Pro?

it doesn't work. it only work with the pro
On Sun, 8 Oct 2023, 2:29 pm Bojangly, @.***> wrote:

It appears twitter has locked this behind Pro. I have tested Basic and it
does not work. Can someone confirm if this works with Pro?
—
Reply to this email directly, view it on GitHub
https://github.com/twitterdev/Twitter-API-v2-sample-code/issues/122#issuecomment-1752029060,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABAYL6UQ4XVJA4EU7T7E33LX6KTC7AVCNFSM6AAAAAAXII6TEGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONJSGAZDSMBWGA
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***
com>


prisma init --with-model

When testing things, it is often needed that after prisma init you add a trivial model to your schema.prisma file. It would be amazing if there was a simple CLI argument that adds a model to the generated file by default, making that step obsolete.
Model can be as simple as our suggestion on prisma generate without a model:
model User {
  id    Int     @id @default(autoincrement())
  email String  @unique
  name  String?
}

Hi @janpio, I'd love to take on this issue.

Hello guys, @fool1280 did you start working on this issue ? If not, can I do the PR ? I worked on it !
Thanks!

Can I step on this if its not taken yet ?.

Hey @Kayoshi-dev , I saw you have raised the PR for this issue, which having merge conflicts, can you plz resolve those, or let me know if i can take over this issue and raise a PR. 👍

Opened a PR to make the required docs changes for this: https://github.com/prisma/docs/pull/5951


Feature/TASK-2_3: Implement Domain-Driven Design

New architecture with DDD
Clean some deprecated codes

Ok to merge!


MappableField with key as raw String

I need to use a raw string for one key because one of the API I'm using (appwrite) is storing the id of a record in the '$id' field.
However, the generated file does not take into account the 'r' before the string and it creates an error and I need to update manually the generated file. I have also tried to escape the $ by adding a \ in front of it but id doesn't not work.
Am I missing something or is there a workaround ?
Thank you
@MappableClass()
class TranslatorModel extends TranslatorEntity with TranslatorModelMappable {
  /// Constructs a [TranslatorModel] instance.
  TranslatorModel({
    @MappableField(key: r'$id') required super.id,
    required super.surname,
    required super.firstname,
    required super.languages,
  });
}

From TranslatorModelMapper :
static const Field<TranslatorModel, String> _f$id =
      Field('id', _$id, key: '$id');

For information, I have implemented the following workaround using a hook on the MappableClass:
class TranslatorMappingHook extends MappingHook {
  const TranslatorMappingHook();
  @override
  Object? beforeDecode(Object? value) {
    if (value is Map<String, dynamic>) {
      // Set 'id' key with r'$id' value
      if (value.containsKey(r'$id')) {
        value['id'] = value[r'$id'];
      }
   }
  }
}

Oh yes seems like a bug, will take a look


Add more GitHub resources

There are lots of resources for learning GitHub. link them here!
https://github.com/mozilla/open-leadership-training-series/blob/gh-pages/_articles/get-your-project-online/project-set-up-for-collaboration-with-github.md
Read the article: https://mozilla.github.io/open-leadership-training-series/articles/get-your-project-online/project-set-up-for-collaboration-with-github/

@acabunoc Can I work on this issue?

Of course! Thanks @Shruti9520

Could Issue #56 be better off as a resource than a section?

PR #59

PR #63


1.3

You have an on-premises application that sends email notifications automatically based on a rule.
You plan to migrate the application to Azure.
You need to recommend a serverless computing solution for the application.
What should you include in the recommendation?
A. a web app
B. a server image in Azure Marketplace
C. a logic app
D. an API app

D

C


Build documentation as Travis step

Our current Travis setup makes it easy to break the documentation without knowing it.  For example right now it's broken because we're using an old psutil api which no longer exists.  This PR fixes the documentation problem and adds a build step so we don't have this issue in the future.

@guidow, PTAL

Looks good to me.


Add basic gateway client

Pull Request Template
Please try to complete the below as best as possible. Some of the fields
may not be necessary so feel free to add or edit as you see fit.
Checklist
Confirm you have completed the following actions prior to submitting this PR.

[ ] There is an existing issue report for this PR.
[ ] I have forked this project.
[x] I have created a feature branch.
[ ] My changes have been committed.
[ ] I have pushed my changes to the branch.

Title
Add support for gateway
New or Changed Features

[x] Connect to the gateway and identify

[x] Heartbeating

[ ] Handle gateway messages

[x] Set heartbeat interval on opcode 10
[x] Require heartbeat ACK to continue
[ ] Sharding
[ ] Resuming
[ ] Dispatch events

[ ] Speedups

[ ] Unit tests (for what we got done)

[ ] Documentation for the gateway package

WIP


fix(tab): fix event handler call

switch event handler call to look at event.isDefaultPrevented() instead of
event.defaultPrevented as angular has implemented that method on jqlite
events.

Fixes #5720

LGTM

Did this not need an associate test change?

@deeg the test was already in there.  the issue is that we don't have separate karma files for testing with and without jquery.  i opened an issue to track that.  also see the thread in our slack for more details.


Lifecycle update and implementations

Updated Overwatch rev
Implemented lifecycle for services:

[ ] Consensus
[ ] http
[ ] log
[ ] mempool
[x] network
[x] storage

I'm not really implementing but leaving the place where we should add features for lifecycle management in services. I'm just making them stop on Shutdown and Kill.
If you feel it is too big, I can leave it here and implement the remaining services in different PRs.

Clippy lint is unrelated. Should be fixed in an external PR


Replace authentication with username and password with OAuth page/token?

It's pretty insecure, but OAuth does the trick. All you have to do really, is setup a page on a website that will ask for some stuff, then it redirects you to OAuth authentication, then login with that?

I don't think Discord has such support 😄 .

At least hide the command from logs.

@nguyenquyhy Discord does support OAuth. It's also been brought to our attention that some of our users aren't happy with supplying their credentials to the server, and would rather use the OAuth system.

@BigJazzz Yeah, it does to be honest.

I'm fairly certain that at this point, Discord's OAuth is only used with bots. I'd love to be proven wrong, though.

It's possible though not probable. You can use discord to login to websites using OAuth but a web application is in place to make that possible. Either a portable web app that server could set up on their own or a centralized application would be needed to really change the login mechanics... And this is a java application not a web application so don't get your hopes up!

Well, people have made all kinds of "web servers" in Minecraft before, like livemaps and various web control panels for plugins. The hardest part, then, is really just that people would have to open ports on their servers to allow the OAuth responses to get through to the server.

@Mohron Well yes, you would have to write a web interface that would route OAuth tokens into the plugin, which would require a second language with something like Ruby on Rails. I understand that you might not want to put some effort into a feature rather than squashing bugs.

which would require a second language with something like Ruby on Rails

Require? I take it you are not fully aware of what Java is capable of?

Yes OAuth2 in Discord is possible.
It requires 2 things:

The plugin to start a webserver for a return page (this is already possible in many plugins/mods such as dynmap).
The server owner to make sure the page is assessable by all its users and proper redirect URI(s) settings for their Discord application, which requires documentation and instructions.

Both are not too hard but would take quite some time. I don't have a lot of time at hand right now, so my temporary solution was to show a warning message when you do /discord login until I have more time or until someone submits a PR :).

@nguyenquyhy So when the login command receives credentials, what happens to them? How are they transmitted? Are they stored anywhere other than our chat log (if it's been enabled)?
I understand it's a time-consuming change, but I would suggest it gets made a priority to avoid someone trying to accuse you of stealing their credentials. That's the whole reason OAuth exists.
For our community, we'd have no problem setting up a webpage and what not, if the plugin was configured to at least require manual work on our end. I understand others may not be in a position where they can do that, but that's a suggestion.

And 2 years on.. still without OAuth token support? lol

Because this project was abandoned by the owner.

Oh well. It seems time has made its' mark.


Subdomain Takeover via Gohire

Gohire.io

Documentation
The CNAME record should be custom.gohire.io which gives the idea that they do not have domain authtification.
Simply create an account and goto hiring setting>career page editor

Hello,
Does this one have another fingerprint?  404 status code, Message error or just NXDOMAIN?

Hi @pdelteil, this is the 404 status code fingerprint


Error hex

please i get this error while running the command
cargo : The term 'cargo' is not recognized as the name of a cmdlet, function, script file, or operable program. Check
the spelling of the name, or if a path was included, verify that the path is correct and try again.
At line:17 char:1

cargo build --release

  + CategoryInfo          : ObjectNotFound: (cargo:String) [], CommandNotFoundException
  + FullyQualifiedErrorId : CommandNotFoundException

copy : Cannot find path 'C:\Users\Administrator\DataSurgeon\target\release\ds.exe' because it does not exist.
At line:24 char:1

copy "$(Get-Location)\target\release\ds.exe" $executablePath

  + CategoryInfo          : ObjectNotFound: (C:\Users\Admini...\release\ds.exe:String) [Copy-Item], ItemNotFoundExce
 ption
  + FullyQualifiedErrorId : PathNotFound,Microsoft.PowerShell.Commands.CopyItemCommand

Hi,
Did this solve your issue? If you don't respond in the next couple of days I'm going to mark this as resolved, but if it's not truly resolved feel free to open a new ticket.
Best,
Drew


Using amqp in a single goroutine that publishes and consumes: am I doing this right?

Hi guys,
Awesome work on the ampq client. I am a Golang n00b and I'm finding it extremely useful.
I am implementing a messaging system that works with websockets and needs to be multi node using your amqp package, and I was wondering if there's a chance you might be able to give me some feedback about my implementation to make sure I am on the right track.
Because I am quite new to both Go and RabbitMQ, even though I have tested my implementation on a multi node use case and it seems to work, I am certain I am doing some things wrong in here and your feedback would be muchly appreciated.
The basic idea is that every node that receives and manages websocket connections (they are stored in a map locally) is also a subscriber on a channel, and upon receiving a message, if the message is intended for a client who's socket connection is missing from the current node, a publish is being sent (using your amqp package) to all other nodes via rabbit mq to check their own local map.
To achieve this I have basically taken your simple consumer/producer examples and have joined them into one rabbit.go file, that has a goroutine that the websockets message hub signals when it wants to publish something. I have also added some retry logic to creating the connection in case it disconnects or fails, and removed the reliable feature as I do not need it badly. I am using the same connection, channel and routing key for all communication (basically just messages between nodes that tell them to look for a connection in the map and push it to the client if it's found).
This is my rabbit.go file which has all the rabbit logic. On my main function which starts things up (the websockets hub etc.) I call the r.run goroutine
func main() {
	flag.Parse()
	go h.run() // socket messages listener
	go r.run() // rabbitmq pubsub listener

	http.HandleFunc("/ws", serveWs) // websockets endpoint
	err := http.ListenAndServe(*addr, nil)
	if err != nil {
		log.Fatal("ListenAndServe: ", err)
	}
}

and then from the hub I am publishing stuff when needed
r.Publish <- sm

Upon receiving a delivery, the consume function just sends the message to the local hub.
This is the entire rabbit.go file:
	package main

	import (
		"encoding/json"
		"flag"
		"fmt"
		"github.com/streadway/amqp"
		"io"
		"log"
		"strings"
		"time"
	)

	var (
		uri          = flag.String("uri", "amqp://guest:guest@localhost:5672/", "AMQP URI")
		exchange     = flag.String("exchange", "signal-exchange", "Durable, non-auto-deleted AMQP exchange name")
		exchangeType = flag.String("exchange-type", "direct", "Exchange type - direct|fanout|topic|x-custom")
		queue        = flag.String("queue", "signal-queue", "Ephemeral AMQP queue name")
		bindingKey   = flag.String("key", "signal-key", "AMQP binding key")
		consumerTag  = flag.String("consumer-tag", "signal-consumer", "AMQP consumer")
		lifetime     = flag.Duration("lifetime", 0*time.Second, "lifetime of process before shutdown (0s=infinite)")
	)

	func init() {
		flag.Parse()
	}

	type Rabbit struct {
		Publish    chan *SockMsgRabbitProxy // this channel receives socket messages from the message hub
		Deliveries <-chan amqp.Delivery     // this channel listens to rabbitmq deliveries and consumes them
		Done       chan error               // this channel receives a message when there the delivery channel breaks
		Channel    *amqp.Channel            // this is the rabbitmq channel we are using to publish and subscribe to messages
	}

	var r = Rabbit{
		Publish: make(chan *SockMsgRabbitProxy),
		Done:    make(chan error),
	}

	func dial(amqpURI []byte) (*amqp.Connection, error) {
		log.Printf("rabbit dialing %q", amqpURI)
		connection, err := amqp.Dial(string(amqpURI))
		if err != nil {
			return nil, fmt.Errorf("Dial: %s", err)
		}
		return connection, nil
	}

	func createChannel(c *amqp.Connection) *amqp.Channel {
		log.Printf("got Connection, getting Channel")
		channel, err := c.Channel()
		if err != nil {
			panic(fmt.Errorf("Channel: %s", err))
		}

		log.Printf("got Channel, declaring %q Exchange (%q)", exchangeType, exchange)
		if err := channel.ExchangeDeclare(
			*exchange,     // name
			*exchangeType, // type
			true,          // durable
			false,         // auto-deleted
			false,         // internal
			false,         // noWait
			nil,           // arguments
		); err != nil {
			panic(fmt.Errorf("Exchange Declare: %s", err))
		}
		return channel
	}

	func createQueue(channel *amqp.Channel, queueName string) (*amqp.Queue, error) {
		log.Printf("declared Exchange, declaring Queue %q", queueName)
		queue, err := channel.QueueDeclare(
			queueName, // name of the queue
			true,      // durable
			false,     // delete when usused
			false,     // exclusive
			false,     // noWait
			nil,       // arguments
		)
		if err != nil {
			return nil, fmt.Errorf("Queue Declare: %s", err)
		}

		log.Printf("declared Queue (%q %d messages, %d consumers), binding to Exchange (key %q)",
			queue.Name, queue.Messages, queue.Consumers, bindingKey)

		if err = channel.QueueBind(
			queue.Name,  // name of the queue
			*bindingKey, // bindingKey
			*exchange,   // sourceExchange
			false,       // noWait
			nil,         // arguments
		); err != nil {
			return nil, fmt.Errorf("Queue Bind: %s", err)
		}
		return &queue, nil
	}

	func getDeliveriesChannel(channel *amqp.Channel, queueName string) (<-chan amqp.Delivery, error) {
		return channel.Consume(
			queueName,    // name
			*consumerTag, // consumerTag,
			false,        // noAck
			false,        // exclusive
			false,        // noLocal
			false,        // noWait
			nil,          // arguments
		)
	}

	// 1. init the rabbitmq conneciton
	// 2. expect messages from the message hub on the Publish channel
	// 3. if the connection is closed, try to restart it

	func (r *Rabbit) run() {
		initRabbitConn()
		for {
			select {
			case sm := <-r.Publish:
				fmt.Printf("publish message: %s\n", sm)
				publish(r.Channel, sm)
			case err := <-r.Done:
				log.Println(err)
				initRabbitConn()
			}
		}
	}

	// try to start a new connection, channel and deliveries channel. if failed, try again in 5 sec.
	func initRabbitConn() {
		ticker := time.NewTicker(5 * time.Second)
		quit := make(chan struct{})
		go func() {
			for {
				select {
				case <-ticker.C:
					if conn, err := dial([]byte(*uri)); err != nil {
						log.Println(err)
						log.Println("node will only be able to respond to local connections")
						log.Println("trying to reconnect in 5 seconds...")
					} else {
						close(quit)
						r.Channel = createChannel(conn)
						createQueue(r.Channel, *queue)
						r.Deliveries, _ = getDeliveriesChannel(r.Channel, *queue)
						go handleConsume(r.Deliveries, r.Done)
					}
				case <-quit:
					ticker.Stop()
					return
				}
			}
		}()
	}

	func publish(c *amqp.Channel, sm *SockMsgRabbitProxy) error {
		body, _ := json.Marshal(sm)
		log.Printf("publishing %dB body (%s)", len(body), body)
		if c == nil {
			return fmt.Errorf("connection to rabbitmq might not be ready yet")
		}
		if err := c.Publish(
			*exchange,   // publish to an exchange
			*bindingKey, // routing to 0 or more queues
			false,       // mandatory
			false,       // immediate
			amqp.Publishing{
				Headers:         amqp.Table{},
				ContentType:     "application/json",
				ContentEncoding: "",
				Body:            []byte(body),
				DeliveryMode:    amqp.Transient, // 1=non-persistent, 2=persistent
				Priority:        0,              // 0-9
				// a bunch of application/implementation-specific fields
			},
		); err != nil {
			return fmt.Errorf("Exchange Publish: %s", err)
		}
		return nil
	}

	func handleConsume(deliveries <-chan amqp.Delivery, done chan error) {
		fmt.Println("waiting for rabbitmq deliveries...")
		for d := range deliveries {
			d.Ack(false)
			s := string(d.Body)
			log.Printf(
				"got %dB delivery: [%v] %s",
				len(s),
				d.DeliveryTag,
				s,
			)
			dec := json.NewDecoder(strings.NewReader(s))
			for {
				var m SockMsgRabbitProxy
				if err := dec.Decode(&m); err == io.EOF {
					break
				} else if err != nil {
					log.Fatal(err)
				}
				fmt.Printf("from: %s, to: %s, broadcast: %t, payload: %s\n", m.From, m.To, m.Broadcast, m.Payload)
				h.send <- &m
			}
		}
		done <- fmt.Errorf("error: deliveries channel closed")
	}

Does this make any sense to you guys?
Thanks for your time and effort! awesome package.

updated route: https://github.com/streadway/amqp/blob/master/_examples/pubsub/pubsub.go

This is a great resource for anyone learning golang and amqp.  Thanks for sharing the fruits of your labor!


Installation instruction

Right now there is no easy way to find the installation instructions without going to RTD. Should we create an INSTALL.md or just link docs/source/install.rst in the README.md

I would just say inside README.md that full documentation including installation instructions is available on RTD.

Done in #455 and #448


View

In this phase of the project, I have added;

A view page to view all users
A show page to show details of each user when a username is clicked
A view page to show details of each post

Thank you
Sent from Yahoo Mail on Android
On Thu, May 18, 2023 at 1:46 PM, Olipliche B.F. Mavoungou @.***> wrote:
@OLIPLICHE approved this pull request.
Hi @Divinity-dev,
Your project is complete! There is nothing else to say other than... it's time to merge it
Congratulations! 🎉
Highlights

Added the views associated with models and controllerd 👏
Verification by running rails c shows the data function as expected 👏

Cheers and Happy coding! 👏 👏 👏
Feel free to leave any questions or comments in the PR thread if something is not 100% clear.
Please, remember to tag me in your question so I can receive the notification.
As described in the Code reviews limits policy you have a limited number of reviews per project (check the exact number in your Dashboard). If you think that the code review was not fair, you can request a second opinion using this form.
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you were mentioned.Message ID: @.***>


⚠ ClamAV has degraded performance

In 7a34984, ClamAV (https://spamassassin.apache.org/) experienced degraded performance:

HTTP code: 200
Response time: 289 ms

Resolved: ClamAV performance has improved in b6d77e7.


Research API's

For now we will basically use two main API's from the Google Maps library set, they're the Google Maps Geocode API and the Google Maps JavaScript API.
Also we will use the UberAPI, @thiagosjk and @Vedolin you folks came to the Uber API talk, so what we can do with the API? What are their limitations?
What we will interact with the UberAPI in order to achieve our goals?
nvm, here is what I'm planning to do with the Google Maps API:

Gathering of Location Data by Address (FTS).
Retrieval of the Data and LatLng attributes (Latitude:Longitude)
Plot the result in the Map.for

The user also could drag to a specific position, that will use the LatLng search pattern of Google Geocoding instead of FTS.
Also, support of the Facebook Developers API for basically doing the following interactions:

Registering the user if it's new (only need Facebook Identifier, E-mail and Full name)
Login if account already exists
Gathering User Friends (Name, Photo, Identifier, E-mail ??)

We can look to initial candidates to the venue with the Text Search API (https://developers.google.com/places/web-service/search?authuser=1&hl=pt-br)
https://maps.googleapis.com/maps/api/place/textsearch/json?query=quadra+de+futebol&location=-23.53013,-46.63409&radius=5000&key={API KEY}
An also Nearby Search
https://maps.googleapis.com/maps/api/place/nearbysearch/json?location=-23.53013,-46.63409&radius=5000&keyword=quadra+de+futebol&key={API KEY}
And the matrix API allow us to calculate several origin and destinations in one shot
https://maps.googleapis.com/maps/api/distancematrix/json?origins=Idexo|Estacao+Santana&destinations=Estacao+Osasco|MASP|Memorial+da+America+Latina&mode=driving&language=en-US


Added cwd_simple segement. It displays the cwd as a single string. Added a new theme yamanpour.

The cwd_simple segment displays the cwd as a single string.

Hey @klyngbaek, thanks for contributing. There was actually a separate pull request that allowed showing the CWD as a single string. It was implemented by passing an option to the existing cwd segment instead of making a new one.
Is the yamanpour theme name something you made up? I just took a brief look at the theme. It looks pretty colorful. I don't really see any reason not to add it. But there is a problem with your implementation that I will point out in a separate comment.
So if you are able to fix up the issue and remove the new segment, I can merge this.


bspacks is broken by adding ocaml-migrate-parsetree

@let-def we're having trouble bspacking ocaml-migrate-parsetree. More specifically, having trouble including the appropriate files after preprocessing with make.
File "migrate_parsetree_versions.ml", line 696, characters 23-42:
Error: Unbound module OCaml_OCAML_VERSION

We're trying to figure how we can preprocess the source files then pack them all together as we do with result, reason, refmt, etc.

Precision: ./bspacks/reason_bspack.sh uses bspack, to which we add -I src/paths, to pack every file into a single file (good for cutting out intermediate tooling, for windows, etc.)

Btw, somewhat related: upgrading bspack flow with #1157

If the version of OCaml is fixed, you can directly run sed with s/OCAML_VERSION/40x/g.
The processed files have the .pp suffix, here you want:
ocaml-migrate-parsetree/_build/default/src/migrate_parsetree_versions.pp.ml

@let-def thanks. After patching things & renaming files & bringing Result into scope (I'm on 4.02.3), I get this:
File "ppx_deriving.ml", line 7, characters 5-24:
Error: Unbound module Ast_convenience_404

Then ppx_tools_versioned is not in scope.

@let-def ok thanks. @tekknolagi and I managed to move until this point now:
File "reason_parser.ml", line 5, characters 2-42:
Error: Unbound value MenhirLib.StaticVersion.require_20160825

A bit lost here. Do you know what's happening? See the updated #1157.

Perhaps we should just get some prebuilt binaries?