Pre-parsed queries and/or Query caching

Right now when you execute against the schema, it parses the queries each time. We could speed up these executions by either parsing some queries beforehand or caching previously seen queries.
The use-case I have in mind is when you'd like to respond to a webhook or expose a REST API that use your GraphQL resolvers internally, having pre-parsed server-side queries would speed up the operation.
Something like this:
func (s *Schema) ExecDocument(
  ctx context.Context,
  query *graphql.Document,
  operationName string,
  variables map[string]interface{},
) *Response {
    // ...
}

An alternative to this is lazily caching the parsed query results, when we encounter the same query twice, use the cached result. This would probably need to be bounded by an LRU cache, but I think it'd be a nice addition to schema.Exec(...)

If you are using GraphQL over HTTP (which I guess is the case for most of the users of this library), then the ideal solution would be to just use old-school Cache-Control headers.
The guys from Apollo have done a great job on caching. They have developed a cache control extension. I do believe that Cache-Control header is the right approach to do it (i.e. CDN/infrastructure agnostic) and  extensions is the way to go. This library would definitely benefit from extensions. Unfortunately, extensions are not supported yet. 😞
To workaround this, in my team we have implemented imperative caching instead of the much better declarative approach. To do this we had to do two things:

Allow GET requests (I'll send a PR for that very soon)
Each resolver can declare it's caching preferences. (maxAge - TTL in seconds and scope - PUBLIC or PRIVATE). Then at the end of each request we set the appropriate CacheControl header for its response. This is not an ideal solution, but a good workaround until this library supports extensions.

Regardless of the caching implementation (imperative/declarative), there is another thing that (usually) goes hand-in-hand with caching - persisted queries. With persisted queries it would e nice to validate queries only once and then every time a persisted query is executed skip the validation step (unless the code has changed).

Hi @pavelnikolov, thanks for the background on apollo's caching system! It's interesting to read what they did to address the stale data problem.
This issue is not exactly about stale data. It's about caching the static queries, so if we see a query come in, we already have the parsed query document ready to go.
In code, something like this:
cache := map[string]*query.Document{}

query := `
  query {
    hero {
      name
      appearsIn
    }
  }
`

cache[query] = query.Parse(query)

So each time we see those static queries again, we have this cache ready with the parsed query.

My point is that with cache hints/imperative caching you'll not have to see the same query again. If you set the cache-control headers, the response will be cached by your infrastructure (Nginx, Varnish, a CDN etc) or in the user's browser.
What you are suggesting would definitely benefit persisted queries, though. With persisted queries, the client application will not have to send the entire query, instead it can send a pre-defined query_id. Also, since the request doesn't need the body, it can be send over GET method. For example:
GET https://your-domain.com/graphql?query_id=5&variales={"key":"value"}

Then the handler can load the query from some cache store (e.g. Redis, Memcached or even embedded datastore). In this case we don't need to validate/parse the query again - we can do it only once when the query is inserted into the datastore.
This is one of the things I would like to work on.
It is possible to use this idea, even without persisted queries or caching. The application would probably store the query+valriables+operation in a key/value store by hash. This way each unique combination of query+variables+operationName will e parsed/validated at most once during the lifetime of the GraphQL application. Is this what you mean?

@pavelnikolov Ahh yep, I see your point now. I guess that's less of an issue with this library and more about how you setup your graphql infrastructure.
I think you're right that cache-control would probably be a better way to architect your app, but I think this is low-hanging fruit to improve the performance of graphql servers out of the box.
This is off-topic, but I've also run into the use case for persisted queries, though in my opinion the other literature online doesn't really do a good job of selling them. From what I've read, they're main benefit is to avoid sending full queries each time. I think an even more important role for them is interacting with existing services. Using your example, rather than:
GET https://your-domain.com/graphql?query_id=5&variables={"key":"value"}

It could just be:
GET https://your-domain.com/user?variables={"key":"value"}

That's what I was getting at with the ability to use Pre-parsed queries

I guess that's less of an issue with this library and more about how you setup your graphql infrastructure

Exactly!
But do you really need this optimisation? Is this actually a bottleneck for your API? I haven't seen the metrics/benchmarks, to be honest, but I think that the time spent there would be relatively insignificant. Maybe there are apps that need this optimisations, though. Just not my team's use-case. I'll think a little bit more about it.

But do you really need this optimisation?

Maybe not, this issue is more about supporting static server-side queries. Seems wasteful to have to parse these kinds of queries since I have access to the schema and query on the server anyway. Then it got me thinking about lazily caching since that would also solve the problem.

This (together with persisted queries) has been on my TODO list for more than a year now.
I’d much rather cache the responses in a CDN, than optimize the library. I agree with you that this is a good optimization and ideally we need both. It’s just a matter of priorities...
Sent from my phone

On 20 Jun 2018, at 20:38, Matthew Mueller<EMAIL_ADDRESS>wrote:
But do you really need this optimisation?
Maybe not, this issue is more about supporting static server-side queries. Seems wasteful to have to parse these kinds of queries since I have access to the schema and query on the server anyway. Then it got me thinking about lazily caching since that would also solve the problem.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

Totally understand! And thanks for the discussion earlier – I learned a lot from this issue :-)


HTTP client.c: fix OSSL_HTTP_proxy_connect() for HTTPS proxy use

I was not happy that many TODOs also for CMP code were removed (during my vacation) in PR #15539 without actually solving them.
I've started tracing those TODOs for which I feel responsible.
I just found out that the answer on the TODO is the server host name correct for TLS via proxy?, which  was removed from apps/cmp.c, is "yes",
but while experimenting on this I encountered a bug that leads to connection failure (with the misleading error: received wrong http version) when the HTTP client tries to CONNECT using TLS via a proxy. This is fixed by the present PR.

Apologies for removing them.  Most didn't seem all that relevant...

Argh, doing the fix I introduced another bug - fixed  now.

Merged - thanks @t8m and @paulidale for your reviews.

BTW, since the pattern for checking if a string is prefixed by a string literal, such as
strncmp(algo, "ecdsa", 5) == 0

is pretty common also elsewhere in OpenSSL code, how about moving
+#define HAS_PREFIX(str, prefix) (strncmp(str, prefix, sizeof(prefix) - 1) == 0)

which I introduced locally in http_client.c, to macros.h and making use of it globally?

Adding that macro more widely seems like a reasonable idea.  I'd modify it slightly:
#define HAS_PREFIX(str, prefix) (strncmp(str, prefix "", sizeof(prefix) - 1) == 0)

this will error if prefix isn't a string constant.  Otherwise badness could happen sizeof(char *) - 1 could walk past the end of a string and will almost certainly not be the desired length.

24 hours has passed since 'approval: done' was set, but as this PR has been updated in that time the label 'approval: ready to merge' is not being automatically set.  Please review the updates and set the label manually.

Closing as this was merged already.


auto play issue.

I'm using this and found the auto play function wasn't working.
It seemed to be a problem with checking whether the slide has an image in it:
obj.has('img') was finding images even though there was none, so the following code wouldn't execute because no image was loaded. I fixed it by changing it to obj.find('img').length.
Line 459
Thought you might want to change it.

It also doesn't seem to work with vertical mode, moving the if interval check for this.auto to the build init seemed to fix this. Not sure if it's the correct way to go about fixing it though.

I have fixed autoplay issue with vertical mode also.
Thank you for pointing this out.


[Housekeeping] Remove CloudIdentity related code in SQL views

As we are no longer checking Cloud Identity assets, some SQL code in the last_compliancestatus view is "dead", and so we should get rid of it.
WITH last_x_days_complianceStatus AS (
    SELECT
        *
    FROM
        `meta-heimdall-dev-003.ram.complianceStatus`
    WHERE
        DATE(_PARTITIONTIME) > DATE_SUB(CURRENT_DATE(), INTERVAL 7 DAY)
        OR _PARTITIONTIME IS NULL
),
assets AS (
    SELECT
        timestamp,
        name,
        owner,
        violationResolver,
        ancestryPathDisplayName,
        ancestryPath,
        ancestorsDisplayName,
        ancestors,
        assetType,
        projectID,
        persistTimeStamp,
        scheduledRootTriggeringID
    FROM
        `meta-heimdall-dev-003.ram.last_assets`
),
latest_asset_inventory_per_rule AS (
    SELECT
        assetName,
        ruleName,
        array_agg(
            struct(assetInventoryTimeStamp, evaluationTimeStamp)
            order by
                assetInventoryTimeStamp desc,
                evaluationTimeStamp desc
        ) [offset(0)] as tms
    FROM
        `meta-heimdall-dev-003.ram.complianceStatus`
    WHERE
        DATE(_PARTITIONTIME) > DATE_SUB(CURRENT_DATE(), INTERVAL 14 DAY)
        OR _PARTITIONTIME IS NULL
    GROUP BY
        assetName,
        ruleName
    ORDER BY
        assetName,
        ruleName
),
latest_rules AS (
    SELECT
        ruleName,
        MAX(ruleDeploymentTimeStamp) AS ruleDeploymentTimeStamp
    FROM
        last_x_days_complianceStatus
    GROUP BY
        ruleName
    ORDER BY
        ruleName
),
status_for_latest_rules AS (
    SELECT
        last_x_days_complianceStatus.*
    FROM
        latest_rules
        INNER JOIN last_x_days_complianceStatus ON last_x_days_complianceStatus.ruleName = latest_rules.ruleName
        AND last_x_days_complianceStatus.ruleDeploymentTimeStamp = latest_rules.ruleDeploymentTimeStamp
),
lastrule_lastasset_compliancestatus AS (
    SELECT
        status_for_latest_rules.evaluationTimeStamp,
        status_for_latest_rules.ruleName,
        REPLACE(
            SPLIT(
                REPLACE(status_for_latest_rules.assetName, "//", ""),
                "/"
            ) [SAFE_OFFSET(0)],
            ".googleapis.com",
            ""
        ) AS serviceName,
        status_for_latest_rules.ruleDeploymentTimeStamp,
        status_for_latest_rules.compliant,
        status_for_latest_rules.assetName,
        status_for_latest_rules.assetInventoryTimeStamp,
        status_for_latest_rules.scheduledRootTriggeringID,
        status_for_latest_rules.deleted,
    FROM
        latest_asset_inventory_per_rule
        INNER JOIN status_for_latest_rules ON status_for_latest_rules.assetName = latest_asset_inventory_per_rule.assetName
        AND status_for_latest_rules.ruleName = latest_asset_inventory_per_rule.ruleName
        AND status_for_latest_rules.evaluationTimeStamp = latest_asset_inventory_per_rule.tms.evaluationTimeStamp
),
enriched_compliancestatus AS (
    SELECT
        lastrule_lastasset_compliancestatus.ruleName,
        lastrule_lastasset_compliancestatus.serviceName,
        REPLACE(
            REPLACE(
                REPLACE(
                    lastrule_lastasset_compliancestatus.ruleName,
                    "ConstraintV1",
                    ""
                ),
                "GCP",
                ""
            ),
            "CI",
            ""
        ) AS ruleNameShort,
        lastrule_lastasset_compliancestatus.ruleDeploymentTimeStamp,
        lastrule_lastasset_compliancestatus.compliant,
        NOT lastrule_lastasset_compliancestatus.compliant AS notCompliant,
        lastrule_lastasset_compliancestatus.assetName,
        lastrule_lastasset_compliancestatus.assetInventoryTimeStamp,
        lastrule_lastasset_compliancestatus.evaluationTimeStamp,
        lastrule_lastasset_compliancestatus.scheduledRootTriggeringID,
        lastrule_lastasset_compliancestatus.deleted,
        assets.owner,
        assets.violationResolver,
        assets.ancestryPath,
        IFNULL(
            assets.ancestryPathDisplayName,
                assets.ancestryPath
        ) AS ancestryPathDisplayName,
        IF(
            ARRAY_LENGTH(assets.ancestorsDisplayName) > 0,
            assets.ancestorsDisplayName,
            assets.ancestors
        ) AS ancestorsDisplayName,
        assets.ancestors,
        assets.assetType,
        assets.projectID,
    FROM
        lastrule_lastasset_compliancestatus
        LEFT JOIN assets ON lastrule_lastasset_compliancestatus.assetName = assets.name
)
SELECT
    enriched_compliancestatus.*,
    SPLIT(
        enriched_compliancestatus.ancestryPathDisplayName,
        "/"
    ) [SAFE_OFFSET(0)] AS level0,
    SPLIT(
        enriched_compliancestatus.ancestryPathDisplayName,
        "/"
    ) [SAFE_OFFSET(1)] AS level1,
    SPLIT(
        enriched_compliancestatus.ancestryPathDisplayName,
        "/"
    ) [SAFE_OFFSET(2)] AS level2,
    SPLIT(
        enriched_compliancestatus.ancestryPathDisplayName,
        "/"
    ) [SAFE_OFFSET(3)] AS level3,
    SPLIT(
        enriched_compliancestatus.ancestryPathDisplayName,
        "/"
    ) [SAFE_OFFSET(4)] AS level4,
    SPLIT(
        enriched_compliancestatus.ancestryPathDisplayName,
        "/"
    ) [SAFE_OFFSET(5)] AS level5,
    SPLIT(
        enriched_compliancestatus.ancestryPathDisplayName,
        "/"
    ) [SAFE_OFFSET(6)] AS level6,
    SPLIT(
        enriched_compliancestatus.ancestryPathDisplayName,
        "/"
    ) [SAFE_OFFSET(7)] AS level7,
    SPLIT(
        enriched_compliancestatus.ancestryPathDisplayName,
        "/"
    ) [SAFE_OFFSET(8)] AS level8,
    SPLIT(
        enriched_compliancestatus.ancestryPathDisplayName,
        "/"
    ) [SAFE_OFFSET(9)] AS level9
FROM
    enriched_compliancestatus
ORDER BY
    enriched_compliancestatus.ruleName,
    enriched_compliancestatus.ruleDeploymentTimeStamp,
    enriched_compliancestatus.compliant,
    enriched_compliancestatus.assetName,
    enriched_compliancestatus.assetInventoryTimeStamp

Hi,
with RAMv1 we where using the Cloud Identity feature, and we still hope to see it back in RAMv2.
As we migrated recently towards RAMv2 we may will work on getting such a (similar) feature back into RAMv2 somewhat later this year. If you would remove this code, it we may just would ask again later to bring it back. :)

hi @MarcFundenberger
Just wanted to know when would _PARTITIONTIME be null?

Hi @yashas224,
In streamed partitioned tables, the _PARTITIONTIME would be NULL for rows that where inserted in the last few minutes.
See https://cloud.google.com/bigquery/docs/querying-partitioned-tables#query_data_in_the_streaming_buffer

Thank you @MarcFundenberger


chore(docker): add HEALTHCHECK

Fixes #1210

Pull Request Test Coverage Report for Build 838389872

0 of 39 (0.0%)  changed or added relevant lines in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-0.06%) to 17.647%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

healthcheck.js
0
39
0.0%

Totals

Change from base Build 838274826:
-0.06%

Covered Lines:
2020

Relevant Lines:
11813

💛  - Coveralls

@chrisns You right, didn't think about that... What's your suggestion? Maybe remove the hardcoded healtcheck and just document it in docker compose?

yeah, I guess so :( or add a config option for it maybe ¯_(ツ)_/¯

Is the healthcheck during setup an issue? Other than the status being set as unhealthy (which makes sense as it is not running as expected) I do not see any negative side affects by default but I will admit I have limited experience here.

@ZetaPhoenix many setups like docker-compose would restart it, you may also find that it wouldn't be included in load balancer configurations where an ingress controller is present

I do not see any negative side affects by default but I will admit I have limited experience here.

The server that keeps restarting it's an issue...

@chrisns Do you think the actual solution is good? Essentially everything is disabled by default, users can choose to enable it when needed

¯_(ツ)_/¯
i'm not really sure what we're trying to achieve + why :)

@chrisns A user asked me to add healthcheck to dockerfile in #1210, I just proposed this as a solution

probably doesn't help things I really strongly disagree with the article they cite :) https://blog.sixeyed.com/docker-healthchecks-why-not-to-use-curl-or-iwr/
yeah this works as a solution, though it's a bit ugly, the reasons in the article don't really stack up IMO and curl remains easier
maintaining curl is installed seems wayyy easier than maintaining a bit of custom code :)

¯_(ツ)_/¯
No idea what to do so


Sometimes emojis (🎙 👋) cause errors

Error occured trying to open in Windows 10 with Git Bash
Simpler to just remove those emojis?

Hmmm, good point, I'll see if I can remove those tonight!


Sogou input method for flutter

Operating environment: Huawei mobile Android 9.0 OS
Description: flutter app, inputting in textfield, pop up soft keyboard, click case letter switch on soft keyboard, the cursor in textfield will move to the beginning  of the textfield.

Hi @AlbertShen0211
Can you please provide your flutter doctor -v,
your flutter run --verbose
and a minimal complete reproducible code sample.
Thank you

flutter doctor -v
[√] Flutter (Channel stable, v1.17.3, on Microsoft Windows [Version 10.0.18363.900], locale zh-CN)
• Flutter version 1.17.3 at D:\Flutter\flutter_windows_1.17.3-stable\flutter
• Framework revision b041144f83 (2 weeks ago), 2020-06-04 09:26:11 -0700
• Engine revision ee76268252
• Dart version 2.8.4
[√] Android toolchain - develop for Android devices (Android SDK version 30.0.0-rc1)
• Android SDK at C:\Users\spb16\AppData\Local\Android\Sdk
• Platform android-30, build-tools 30.0.0-rc1
• ANDROID_HOME = C:\Users\spb16\AppData\Local\Android\Sdk
• Java binary at: C:\Program Files\Android\Android Studio\jre\bin\java
• Java version OpenJDK Runtime Environment (build 1.8.0_242-release-1644-b01)
• All Android licenses accepted.
[√] Android Studio (version 4.0)
• Android Studio at C:\Program Files\Android\Android Studio
• Flutter plugin version 46.0.2
• Dart plugin version 193.7361
• Java version OpenJDK Runtime Environment (build 1.8.0_242-release-1644-b01)
[√] Connected device (1 available)
• VKY AL00 • WJX7N17905004130 • android-arm64 • Android 9 (API 28)

Hi @AlbertShen0211
thanksfor your doctor,
could you please provide the remaining information listed above
Thank you

Without additional information we are unfortunately not sure how to resolve this issue.
We are therefore reluctantly going to close this bug for now.
Please don't hesitate to comment on the bug if you have any more information for us; we will reopen it right away!
Thanks for your contribution.


Refactor the storage subsystem to support rust async io

Current nydus uses sync IO with multi-threading, which may uses too many threads. It would be great to switch to rust async IO model. We could do it step by step and start with the storage subsystem.
A proposed model as below

There is still one puzzle missing, read_vectored_at_volatile() is sync:(
With https://github.com/cloud-hypervisor/fuse-backend-rs/pull/81 merged, we could use async_read_vectored_at_volatile to solve the last puzzle.


Table 38 Purchase Header - [EventRequest] "OnValidatePayToVendorNoOnAfterConfirmed"

Describe the request
Greetings,
I'd like to know if it's possible to add the event "OnValidatePayToVendorNoOnAfterConfirmed" to table 38 "Purchase Header".
The event would be similar to the event "OnValidateBillToCustomerNoOnAfterConfirmed" of table 36 "Sales Header".
Best Regards,
Jo??o Rosa
Additional context
We would like to extend some custom sales functionality into purchases.
Internal work item: AB#540613

Thanks for reporting this. We agree, and we’ll publish a fix asap, either in an update for the current version or in the next major release. Please do not reply to this, as we do not monitor closed issues. If you have follow-up questions or requests, please create a new issue where you reference this one.
Build ID: 22165.


Channels should support keys for labeling outcomes during simulations

there should be a way to get results out during simulation

Adding measurement_key protocol.

There is a concrete proposal to move to a protocol _acts_on_ and this would be fixed as part of this.

Absorbing into #3566 to capture more recent discussion.

Duplicate of #3566


the new version does not work

That's because this is a tool to generate templates. You can either use the legacy version from the legacy directory or what I would reccomend in the mean time is use one of these more up to date templates
https://github.com/Rabbitminers/Create-Multiloader-Addon-Template
https://github.com/Fabricators-of-create/Create-Multiloader-Addon-Template


feat(net): add "helm.sh/..." annotation on lb deployment

What this PR does / why we need it:
Add your description
Which issue(s) this PR is related to (optional, link to 3rd issue(s)):
Fixes #
Reference to #

Special notes for your reviewer:
/cc @hicaistar @whalecold

Release note:

NONE

/cherrypick release-0.5

LGTM
@hicaistar

/lgtm

this PR is for https://github.com/caicloud/prod-issue/issues/931


meta-ros-common: add a warning message about *-draft branch being deprecated

The same change pushed to zeus-draft, warrior-draft and thud-draft branches as well (without separate PRs).


No issue - just a massive THANK YOU

Just wanted to say THANK YOU for this. This tool was my primary method of checking results cause it was so easy to know where things stood without loading giant, bloated news sites. I know there were many more like me that used this.
Thank you thank you thank you - to everyone who contributed!
@alex
@frewsxcv
@thomasdesr
@fractionalhare
@samczsun
@titanous
@bojanrajkovic
@JulesDT
@bellachp
@saleemrashid
...and the other 40 kind folks who made PRs and commits.

Yeah that was a convenient tool.
Greetings from Germany. That race was called(new President('Joe Biden')) to the end.

Yes, thank you!

You guys did an excellent job! Thank you!

I wanted to do this same thing last night, but didnt want to mess with the construciton process. Thank you guys, you did an amazing job with this one!

I run a data/metrics shop at a big financial institution and have an appreciation for what you did.
Awesome job.
Thank you so much!

It's a wonderful app!  Thanks so much for your work!

We're thrilled it's been so useful to so many people!

Its still useful. Until all the votes are in and certified, I'm keeping this open in a tab. Thank you again!


Survival information from TCGA RNA seq

How did you figure out patients' survival information (time to event) from TCGA RNA-seq data? There seems to be no feature indicating survival or censorship.

There is another file related to the RNA-seq data


[SPARK-5284][SQL] Insert into Hive throws NPE when a inner complex type field has a null value

JIRA: https://issues.apache.org/jira/browse/SPARK-5284

Test build #25678 has started for   PR 4077 at commit fceacd6.

This patch merges cleanly.

Test build #25678 has finished for   PR 4077 at commit fceacd6.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/25678/
Test PASSed.

Thanks! I'm merging to master.


e2e: Add debug steps to verify the cert with CA

This is just for debug purpose to understand what is actually happen when oc login require --insecure-skip-tls-verify.

/hold

/unhold
@adrianriobo, let's start using that to check why the login failure happen.

:thinking: ok but not sure if we want to keep these steps on our e2e scenario, I can just create a custom version for e2e container image with these steps in and try to reproduce on downstream so may keep the hold and once I got the version we can close the PR WDYT?

What is/was the login failure? What was achieved/understood thanks to this additional debug code?

This is related to https://github.com/crc-org/crc/issues/3741


Tower_setup.yml Fails with Certificate Issue

SUMMARY

Failed to update inventory: Could not establish a secure connection. Please add the server to your certificate authority.
You can run this command without verifying SSL with the --insecure flag, or permanently disable verification by the config setting:
 tower-cli config verify_ssl false

ISSUE TYPE

Bug Report

EXTRA VARS FILE

None

e.g. here is an example of an extra vars file we are looking for:
$ cat ~/Github/linklight/provisioner/seans_workshop.yml
---
ec2_region: us-east-1
ec2_name_prefix: seantest
student_total: 25
admin_password: ansible
create_login_page: true

for more information on the extra vars file please refer to: https://github.com/network-automation/linklight/blob/master/provisioner/README.md
ANSIBLE VERSION

ansible 2.8.0.dev0

CONFIGURATION

DEFAULT_HOST_LIST(/home/student1/.ansible.cfg) = [u'/home/student1/networking-workshop/lab_inventory/hosts']
DEFAULT_STDOUT_CALLBACK(/home/student1/.ansible.cfg) = yaml
DEFAULT_TIMEOUT(/home/student1/.ansible.cfg) = 60
DEPRECATION_WARNINGS(/home/student1/.ansible.cfg) = False
HOST_KEY_CHECKING(/home/student1/.ansible.cfg) = False
PERSISTENT_COMMAND_TIMEOUT(/home/student1/.ansible.cfg) = 60
PERSISTENT_CONNECT_TIMEOUT(/home/student1/.ansible.cfg) = 60
RETRY_FILES_ENABLED(/home/student1/.ansible.cfg) = False

OS / ENVIRONMENT

Fedora 29 is the local machine, but SSH to the AWS Workshop lab
TOWER

PLAYBOOK SHORT OUTPUT

TASK [CREATE INVENTORY] **************************************************************************************************************************
task path: /home/student1/networking-workshop/tower_setup.yml:9
<<IP_ADDRESS>> ESTABLISH LOCAL CONNECTION FOR USER: student1
<<IP_ADDRESS>> EXEC /bin/sh -c 'echo ~student1 && sleep 0'
<<IP_ADDRESS>> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/student1/.ansible/tmp/ansible-tmp-1549858377.34-223452302866736 `" && echo ansible-tmp-1549858377.34-223452302866736="` echo /home/student1/.ansible/tmp/ansible-tmp-1549858377.34-223452302866736 `" ) && sleep 0'
Using module file /usr/lib/python2.7/site-packages/ansible/modules/web_infrastructure/ansible_tower/tower_inventory.py
<<IP_ADDRESS>> PUT /home/student1/.ansible/tmp/ansible-local-11820YgbEKY/tmpTdz3Pk TO /home/student1/.ansible/tmp/ansible-tmp-1549858377.34-223452302866736/AnsiballZ_tower_inventory.py
<<IP_ADDRESS>> EXEC /bin/sh -c 'chmod u+x /home/student1/.ansible/tmp/ansible-tmp-1549858377.34-223452302866736/ /home/student1/.ansible/tmp/ansible-tmp-1549858377.34-223452302866736/AnsiballZ_tower_inventory.py && sleep 0'
<<IP_ADDRESS>> EXEC /bin/sh -c '/usr/bin/python /home/student1/.ansible/tmp/ansible-tmp-1549858377.34-223452302866736/AnsiballZ_tower_inventory.py && sleep 0'
<<IP_ADDRESS>> EXEC /bin/sh -c 'rm -f -r /home/student1/.ansible/tmp/ansible-tmp-1549858377.34-223452302866736/ > /dev/null 2>&1 && sleep 0'
The full traceback is:
WARNING: The below traceback may *not* be related to the actual failure.
  File "/tmp/********_tower_inventory_payload_P0LiWD/__main__.py", line 111, in main
    org = org_res.get(name=organization)
  File "/usr/lib/python2.7/site-packages/tower_cli/models/base.py", line 499, in get
    response = self.read(pk=pk, fail_on_no_results=True, fail_on_multiple_results=True, **kwargs)
  File "/usr/lib/python2.7/site-packages/tower_cli/models/base.py", line 301, in read
    r = client.get(url, params=params)
  File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 476, in get
    return self.request('GET', url, **kwargs)
  File "/usr/lib/python2.7/site-packages/tower_cli/api.py", line 257, in request
    r = self._make_request(method, url, args, kwargs)
  File "/usr/lib/python2.7/site-packages/tower_cli/api.py", line 165, in _make_request
    'Could not establish a secure connection. '

fatal: [ansible]: FAILED! => changed=false
  invocation:
    module_args:
      description: null
      host_filter: null
      kind: ''
      name: Workshop Inventory
      organization: Default
      state: present
      variables: null
  msg: |-
    Failed to update inventory: Could not establish a secure connection. Please add the server to your certificate authority.
    You can run this command without verifying SSL with the --insecure flag, or permanently disable verification by the config setting:

     tower-cli config verify_ssl false

PLAYBOOK LONG OUTPUT

[student1@ansible networking-workshop]$ ansible-playbook tower_setup.yml  -vvvv
ansible-playbook 2.8.0.dev0
  config file = /home/student1/.ansible.cfg
  configured module search path = [u'/home/student1/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python2.7/site-packages/ansible
  executable location = /usr/bin/ansible-playbook
  python version = 2.7.5 (default, Sep 12 2018, 05:31:16) [GCC 4.8.5 20150623 (Red Hat 4.8.5-36)]
Using /home/student1/.ansible.cfg as config file
setting up inventory plugins
host_list declined parsing /home/student1/networking-workshop/lab_inventory/hosts as it did not pass it's verify_file() method
script declined parsing /home/student1/networking-workshop/lab_inventory/hosts as it did not pass it's verify_file() method
auto declined parsing /home/student1/networking-workshop/lab_inventory/hosts as it did not pass it's verify_file() method
Parsed /home/student1/networking-workshop/lab_inventory/hosts inventory source with ini plugin
Loading callback plugin yaml of type stdout, v2.0 from /usr/lib/python2.7/site-packages/ansible/plugins/callback/yaml.pyc

PLAYBOOK: tower_setup.yml ************************************************************************************************************************
Positional arguments: tower_setup.yml
become_user: root
become_method: sudo
inventory: (u'/home/student1/networking-workshop/lab_inventory/hosts',)
tags: (u'all',)
forks: 5
verbosity: 4
connection: smart
timeout: 60
1 plays in tower_setup.yml

PLAY [TOWER CONFIGURATION WITH PLAYBOOKS] ********************************************************************************************************
META: ran handlers

TASK [CREATE INVENTORY] **************************************************************************************************************************
task path: /home/student1/networking-workshop/tower_setup.yml:9
<<IP_ADDRESS>> ESTABLISH LOCAL CONNECTION FOR USER: student1
<<IP_ADDRESS>> EXEC /bin/sh -c 'echo ~student1 && sleep 0'
<<IP_ADDRESS>> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/student1/.ansible/tmp/ansible-tmp-1549858377.34-223452302866736 `" && echo ansible-tmp-1549858377.34-223452302866736="` echo /home/student1/.ansible/tmp/ansible-tmp-1549858377.34-223452302866736 `" ) && sleep 0'
Using module file /usr/lib/python2.7/site-packages/ansible/modules/web_infrastructure/ansible_tower/tower_inventory.py
<<IP_ADDRESS>> PUT /home/student1/.ansible/tmp/ansible-local-11820YgbEKY/tmpTdz3Pk TO /home/student1/.ansible/tmp/ansible-tmp-1549858377.34-223452302866736/AnsiballZ_tower_inventory.py
<<IP_ADDRESS>> EXEC /bin/sh -c 'chmod u+x /home/student1/.ansible/tmp/ansible-tmp-1549858377.34-223452302866736/ /home/student1/.ansible/tmp/ansible-tmp-1549858377.34-223452302866736/AnsiballZ_tower_inventory.py && sleep 0'
<<IP_ADDRESS>> EXEC /bin/sh -c '/usr/bin/python /home/student1/.ansible/tmp/ansible-tmp-1549858377.34-223452302866736/AnsiballZ_tower_inventory.py && sleep 0'
<<IP_ADDRESS>> EXEC /bin/sh -c 'rm -f -r /home/student1/.ansible/tmp/ansible-tmp-1549858377.34-223452302866736/ > /dev/null 2>&1 && sleep 0'
The full traceback is:
WARNING: The below traceback may *not* be related to the actual failure.
  File "/tmp/********_tower_inventory_payload_P0LiWD/__main__.py", line 111, in main
    org = org_res.get(name=organization)
  File "/usr/lib/python2.7/site-packages/tower_cli/models/base.py", line 499, in get
    response = self.read(pk=pk, fail_on_no_results=True, fail_on_multiple_results=True, **kwargs)
  File "/usr/lib/python2.7/site-packages/tower_cli/models/base.py", line 301, in read
    r = client.get(url, params=params)
  File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 476, in get
    return self.request('GET', url, **kwargs)
  File "/usr/lib/python2.7/site-packages/tower_cli/api.py", line 257, in request
    r = self._make_request(method, url, args, kwargs)
  File "/usr/lib/python2.7/site-packages/tower_cli/api.py", line 165, in _make_request
    'Could not establish a secure connection. '

fatal: [ansible]: FAILED! => changed=false
  invocation:
    module_args:
      description: null
      host_filter: null
      kind: ''
      name: Workshop Inventory
      organization: Default
      state: present
      variables: null
  msg: |-
    Failed to update inventory: Could not establish a secure connection. Please add the server to your certificate authority.
    You can run this command without verifying SSL with the --insecure flag, or permanently disable verification by the config setting:

     tower-cli config verify_ssl false

PLAY RECAP ***************************************************************************************************************************************
ansible                    : ok=0    changed=0    unreachable=0    failed=1    skipped=0

[student1@ansible networking-workshop]$ ^C
[student1@ansible networking-workshop]$ ^C
[student1@ansible networking-workshop]$

Workaround:
For now I have been able to get this to work by adding:
tower_verify_ssl: false
to each Play that calls https://localhost
Should this be able to be cast globally=?  Is there a workaround to get ssl to work without this workaround.

going to have this conversation here: https://github.com/network-automation/linklight/pull/177


Set expiration on a lock row not represented in redis

I am performing a simple lock on a redis object
lock = Redis::Lock.new('foobar', expiration: 60)
lock.lock do
  puts "In my custom lock"
end

And when I review the row I do not see the TTL set when I review in the redis-cli
<IP_ADDRESS>:6379> ttl foobar
(integer) -1

Your test exits the lock too quickly.  Here's how you can see it in action:

Your test exits the lock block too quickly.  Use sleep so you can see it in action:
lock = Redis::Lock.new('foobar', expiration: 60)
lock.lock do
  sleep 30
end

Within 30 seconds, check the TTL:
<IP_ADDRESS>:6379> ttl 'foobar'
(integer) 59

After 30 seconds, check the TTL again and it will be gone:
<IP_ADDRESS>:6379> ttl 'foobar'
(integer) -2


Update example in README.rst

The original example doesn't call client.close() to release resources properly.
Since client supports context manager, using with will ensure resources are released when we are done with client object.

Although threads spawned by Client constructor will be reaped on program exit (equivalent to client.close()), emphasizing the context manager pattern makes sense for longer running programs.

Thanks @randomir!


Prediction for new set of premise and hypothesis pair

I am having very hard time in understanding how to predict for a new set of premise, hypothesis pair?? #help

The usage of the model is explained in the repository README in details. I'm not sure what your issue is here exactly ? To make predictions on premise/hypothesis pairs, they need to be formatted like the test data of the SNLI or MNLI data sets and fed to the model with the test_snli.py or test_mnli.py scripts.

that means I have to parse those hypothesis and premise sentences with some kind of parser because the snli_1.0_test file contains parsed sentences too.

Yes, out of convenience, I used in my scripts the sentences that had already been parsed in the SNLI and MNLI data sets. This means that, if you want to reuse the train or test scripts for snli or mnli, you will need to parse your data set's sentences first. This is however rather easy to do in Python. You could also simply use the Stanford PCFG Parser, which is what was actually used to produce the pre-parsed sentences in the SNLI and MNLI data sets.

Okay... Thank you..


SSHKit::Runner::ExecuteError: Exception while executing as<EMAIL_ADDRESS>No digester supplied

I've set up capistrano this way: https://gorails.com/deploy/ubuntu/16.04
Using the latest versions of every gem.
What I get when I want to do " cap production deploy" is the following:
"(Backtrace restricted to imported tasks)
cap aborted!
SSHKit::Runner::ExecuteError: Exception while executing as<EMAIL_ADDRESS>No digester supplied
Caused by:
No digester supplied
Tasks: TOP => rbenv:validate
(See full trace by running task with --trace)"
So I'm quite new in coding and I still need to learn things. Can someone tell me what's going with this message. I couldn't find anything thru Google. It's mysterious. If you need more details please let me know.

Can you provide more detail by running cap production deploy --trace as suggested by the error message? That will help us see where the exception is being raised.


im_names referenced before assignment

(pose) D:\PYTHON\pose_detection>python videopose.py
outputs/dance.mp4 --- elapsed time: 3.8129527000000003 s
Traceback (most recent call last):
File "videopose.py", line 164, in
inference_video('outputs/dance.mp4', 'alpha_pose')
File "videopose.py", line 160, in inference_video
main(args)
File "videopose.py", line 69, in main
keypoints = detector_2d(video_name)
File "D:\PYTHON\pose_detection\joints_detectors\Alphapose\gene_npz.py", line 67, in handle_video
data_loader = ImageLoader(im_names, batchSize=args.detbatch, format='yolo').start()
UnboundLocalError: local variable 'im_names' referenced before assignment
Do you have any idea why this is happens ?

The im_names is not properly initialized at here. Try to set a breakpoint there and check if there is any bug caused by typo in video path.


"Loading spinner" only displays the very first time it's rendered

I used "the big button" to turn off the services and i caught a glimpse of the wait spinner as it turned back on:

however this is the only time that it will be shown. subsequent on/offs will not show the wait spinner

fixed with 1.5.x release stream


Powershell not parse mixed single and double quotes [found via: -command Set-Location]

Steps to reproduce

execute:

>powershell.exe -noexit -command Set-Location "X:\I'll do"

Expected behavior
open powershell in "X:\I'll do" location
Actual behavior
powershell return:
The string is missing the terminator: '.
+ CategoryInfo          : ParserError: (:) [], ParentContainsErrorRecordException
+ FullyQualifiedErrorId : TerminatorExpectedAtEndOfString
Environment data

> $PSVersionTable

Name                           Value
----                           -----
CLRVersion
PSRemotingProtocolVersion      2.3
SerializationVersion           <IP_ADDRESS>
PSVersion                      6.0.0-alpha
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
GitCommitId                    v6.0.0-alpha.14
PSEdition                      Core
WSManStackVersion              3.0
BuildVersion                   <IP_ADDRESS>

i think it's not a bug.
$a = "A:\pscode\temp184\aaa'b"
cd -LiteralPath $a
cd ..
pwd
$b = "aaa'b"
cd -LiteralPath $b
pwd
get-command -ParameterName LiteralPath
Merry Christmas to every American,happy new year to you.

i think that the problem is from System.Management.Automation
a tokenizer problem (i'm not a C# programer, sorry if fail)

And Merry Christmas and new year.

This is not a PowerShell bug - the command line you've passed will not contain the double quotes.
C:\>echoargs -noexit -command Set-Location "X:\I'll do"
arg 0: <-noexit>
arg 1: <-command>
arg 2: <Set-Location>
arg 3: <X:\I'll do>

CommandLine:
echoargs  -noexit -command Set-Location "X:\I'll do"

Notice that arg 3 is a single argument (because you used quotes), but the actual argument does not contain the quotes.
This is how argument parsing works.
PowerShell tries to be help and concatenates all of the argument after -Command, but in cases like this, it can be a bit confusing because values get processed by the shell you use (in your case cmd) before PowerShell gets a chance to parse the concatenated command line.
Here is one possible solution (assuming your shell is cmd, it may work in other shells as well):
powershell -noexit -command "Set-Location \"X:\I'll do\""

thanks @lzybkr


Allow List of Addresses in Filter

Expected Behavior
Filters are used to track changes (e.g. new logs from a given contract address). According to the JSON-RPC spec for eth_newFilter, the address field should be:

address: DATA|Array, 20 Bytes - (optional) Contract address or a list of addresses from which logs should originate.

The address is allowed to be either:

null (to watch everything)
a single address to watch
an array of addresses to watch

Current Behavior
Currently in ganache, filters work for null or for a single address. When an array of addresses is passed in to eth_newFilter to ganache-core, the resulting filter always return blank (no matches). This is in violation of the spec above.
Possible Solution
StateManager.prototype.getLogs should allow filter.address to be an array and handle that case appropriately. This should be a minor change.
Steps to Reproduce (for bugs)

Deploy any contract with any public function which produces a log (event)
Create a new filter with address=<address from above>
View logs associated with that filter
Created a new filter with address=[<address from above>] (that is, an array)
See no logs associated with that filter

Context
Implementing JSON-RPC / web3 correctly is an important goal for ganache and Truffle.
Your Environment

Version used: Truffle v4.0.1 (core: 4.0.1)

This was fixed in https://github.com/trufflesuite/ganache-core/pull/130


Add IRC Channel to connect with community

Can we please add links to IRC channel in the project idea along with mentor information?
This would be very helpful to directly go to IRC channel and ask questions to the community.

Can we please add links to IRC channel in the project idea

Most CNCF projects have their own communication means. For example, Kubernetes has it's own slack instance (https://slack.k8s.io/). https://www.cncf.io/projects/ contains info about communication methods for all projects.
For broad CNCF related discussions, you can also join the CNCF slack at https://slack.cncf.io/. :)

along with mentor information

Mentors are listed along with the project idea, you should be able to ping them directly once you join the projects' slack.

Closing, feel free to comment here if you have any more questions about this.


Can't get inline image from Gmail API

Hello,
<?php // ... $mime = ZBateson\MailMimeParser\Message::from(Base64Url::decode($mail->getRaw())); echo $mime->getHtmlContent();
I'm getting the right mail, with HTML like i want, but images sources are replaced by : cid:image.png.
I understand that they put images in attachments, but can't get a way to make it work.

Hi @Lepelley --
The cid URL scheme points to an attachment with the given Content-Id header.  You can read more about it here: https://tools.ietf.org/html/rfc2111
Although the actual "replacement" of the image with content is beyond the scope of a mail parser (you may want to inline it, serve it as a URL, display it in a window or you just want to look at the html content), you can find parts with a given Content-Id easily:
$message = Message::from(...);
$part = $message->getPartByContentId('the content ID');

For an example, although I'm doing it a bit differently (I wanted to use the attachment's index to retrieve a part rather than the Content-Id), you can check out how I replaced it in here:
https://github.com/zbateson/mailbox-folder/blob/ef013a942811d1f4dafb7ae905afa812b9c905b7/src/App/Actions/EmailRestAction.php#L36
In that example it inserts a link to /emails/{id}/attachments/{att_id}/{filename}?inline=true, which causes this action:
https://github.com/zbateson/mailbox-folder/blob/master/src/App/Actions/EmailAttachmentAction.php
Hope that helps!

Helps a lot, thank you !

Hey,
sorry for commenting on this old post. Maybe my info is already outdated, but I found this thread via google and it pointed me to finding this rather easy solution for the "problem beyond scope" of embedding cid scheme attached pictures in the HTML output:
  $content = $message->getHtmlContent();
  $domDocument = new DOMDocument();
  libxml_use_internal_errors(true); //mute warnings of non well formated $content
  $domContent = $domDocument->loadHTML($content);
  $images = $domDocument->getElementsByTagName('img');
  foreach ($images as $image)
  {
    //image links are parsed to find out whether they are embedded or external:
    $srcLink = $image->getAttribute('src');
    $components = parse_url($srcLink);
    if($components !== false)
    {
      if(strtolower($components['scheme'] == 'cid'))
      {
        //this is handling of embedded images as attachments
        $contentId = $components['path'];
        $part = $message->getPartByContentId($contentId);
        $imgContent = $part->getContent();
        $imgContentType = $part->getHeaderValue(HeaderConsts::CONTENT_TYPE);
        //convert attachment to direct embedded base64 data:
        $image->setAttribute('src', 'data:' . $imgContentType . ';base64,' . base64_encode($imgContent));
      }
      elseif(strtolower($components['scheme'] == 'data'))
      {
        //do nothing, this is handling of embedded images, e.g.:
        //<img src="data:image/jpg;base64,{{base64-data-string here}}" />
        //by leaving it as it is => will already be properly displayed!
      }
      else
      {
        //handling of external links beginning with e.g. http/https (in my case I wanted to remove them)
        $image->setAttribute('src', ''); //by removing image links
      }
    }
    else
    {
      //in case parsing of src url fails:
      $image->setAttribute('src', ''); //also remove image link
    }
  }
  echo $domDocument->saveHTML();

My solution basically just parses the HTML and replaces cid references with the data from the attachment. Am I missing some edge cases here, or is it really that easy? If so, wouldn't this be a feasible solution for your library as well, @zbateson ?
Best regards,
Didi

Hi @didi1357 --
Thanks for that -- I'm sure people will find it helpful searching like you have :)
Beyond scope meaning I don't think it belong in the library.  It's certainly adjacent and very close, but not related to the actual parsing and reading of email content.  I could see it be desirable to an 'email client' that uses an email parser to read emails and also displays the email to an end user, but there could be many 'very close' things (like maybe finding the 'reply' portion of an email, stripping out the rest) that aren't strictly 'parsing' and should be in separate libraries.
All the best and thanks again!


Add Combined mutipathogen plot

Add code to create the "combined multipathogen" plot with the possibility to have multiple pathogens (no limited to 2) and age group filtering

Done in #6


KVM, libvirt, VirtManager audio support

Issue description
I have installed Windows XP on KVM using Virtual Machine Manager.
I have selected ES1370 audio device as described here: https://www.linux-kvm.org/page/Sound
Virtual audio device is detected by Windows and audio can be played back, but i don't hear it.

I use GNOME3 desktop with pulseaudio.
I would expect that it just works (in best case). Can it be fixed or do i have to configure anything manually in addition?
Steps to reproduce

enable KVM

  virtualisation.libvirtd = {
    enable = true;
    qemuPackage = pkgs.qemu_kvm;
  };

put user in group libvirtd
install package virtmanager
connect to local hypervisor
install OS
play audio

Technical details

system: "x86_64-linux"
host os: Linux 4.14.81, NixOS, 18.09.1363.b37872d4268 (Jellyfish)
multi-user?: yes
sandbox: yes
version: nix-env (Nix) 2.1.3
channels(davidak): ""
channels(root): "nixos-18.09.1363.b37872d4268, nixos-hardware, nixos-unstable-19.03pre159472.80738ed9dc0"
nixpkgs: /nix/var/nix/profiles/per-user/root/channels/nixos

I've used a Xubuntu guest with the ICH6 audio and PulseAudio on the host and it works fine. Do you see a  "playback" stream from virt-manager in pavucontrol?

@emmanuelrosa no

I installed elementary OS with ICH6 audio and it works.

But when i use that with ES1370, it also don't have sound. So i think ES1370 is broken.
(I figured out that AC97 audio works with Windows XP out of the box)


Make sure IHttpContextAccessor is always registered

The IHttpContextAccessor allows access to the current executing HttpContext and seems to be a core component of ASP.NET. Still however, the IHttpContextAccessor is not registered by default by ASP.NET while it IMO definitely should.
The whole idea behind the new configuration model in ASP.NET is that everything is registered for you magically behind the covers, but even if you call AddMvc or even AddMvcCore the IHttpContextAccessor isn't registered, while there are many integration scenarios that require its availability. This means that 3rd parties (or possibly even future framework components) always need to check its availability and throw an exception when it's not registered and application developers always need to register it manually.

The whole idea behind the new configuration model in ASP.NET is that everything is registered for you magically behind the covers, but even if you call AddMvc or even AddMvcCore the IHttpContextAccessor isn't registered, while there are many integration scenarios that require its availability. This means that 3rd parties (or possibly even future framework components) always need to check its availability and throw an exception when it's not registered and application developers always need to register it manually.

Not everything, everything that is required. We did extra work to make sure that our own types don't rely on it (because ambient state is bad and hard to reason about).
One of the reasons we took it out for performance. We don't want every ASP.NET application to enable execution context copies (what happens as soon as you use async local). Also, we don't really want users to use this type unless they need it.

Though to be honest, if nobody resolves it, there should be no cost.

One of the reasons we took it out for performance.

I find it hard to believe that the use of such Ambient Context will actually cause any detectable performance hit. Especially with all the I/O going on. Have you actually measured this and publically described this somewhere what the performance impact is?

we don't really want users to use this type unless they need it.

I don't believe that adding this by default will cause users to automatically use it. I just think that not adding it by default will simply cause users to add it by hand.

because ambient state is bad

Apparently not that bad, because ASP.NET uses it for logging scopes.

every await and dispatch to threadpool is no longer on the default execution context fast path.

Since everybody uses logging in their application, doesn't this mean that we never run on this context fast path?

because ambient state is bad

Apparently not that bad, because ASP.NET uses it for logging scopes.

Only ConsoleLogScope alters the execution context in this way

Ah yes that was the problem with turning it on by default. We need to resolve it ourselves to set it. It needs to be added by the components that need it.

I find it hard to believe that the use of such Ambient Context will actually cause any detectable performance hit. Especially with all the I/O going on. Have you actually measured this and publically described this somewhere what the performance impact is?

Yes, @benaadams measured it as part of our benchmarks runs and it was significant. Scoped logging does tend to use async local as well so the cost might be incurred there depending on the logging system. These things are fine, they just can't be all on by default.

Yes, @benaadams measured it as part of our benchmarks runs and it was significant.

Can you link to his benchmark results?

This was the change that disabled it.

First the benchmark - https://github.com/aspnet/benchmarks/commit/909a929d2b37a57934b92de1ef760d55f3b65d43
Officially removed- https://github.com/aspnet/HttpAbstractions/pull/475

Can't find any performance numbers on the repo but you might be able to the differences in the excel spreadsheet in the Updated Results commit after the change:
https://github.com/aspnet/benchmarks/commits/dev?after=G6n%2FgPpHkWvHSt0NmAuyU1ZqXqIrMTM5

Have you actually measured this and publically described this somewhere what the performance impact is?

https://github.com/aspnet/Hosting/issues/330
https://github.com/aspnet/Hosting/issues/356
On 4.6+ it allocates 680k per 2000 requests and creates an addtional dictionary per request
Every await that is async allocates an additional 8 bytes https://github.com/dotnet/coreclr/pull/3157
When the HttpContext is set ExecutionContext.SetLocalValue is additionally run

You don't normally need to use IHttpContextAccessor as DI will inject the HttpContext for you if you ask for it.

You don't normally need ambient state. It shouldn't be the default way of doing things. This is an  optional convenience for people. Unfortunately it's required to get the scoped service provider for scenarios like this https://github.com/aspnet/Mvc/issues/5403

Moar perf numbers - https://github.com/aspnet/Mvc/issues/3523

What I see is that the there is especially a perf hit under .NET 4.5.1 where there is no optimized AsyncLocal. Besides that, we will see the improvement be deminished when any I/O is done. But since this optimization is a very conscious action, I understand that you won't go to the 'default on' scenario. It's unfortunate though that this makes less intuitive for non-conformers.

@dotnetjunkie It shouldn't be exposed if you add the right services. services.AddSimpleInjectorServices()

It shouldn't be exposed if you add the right services. services.AddSimpleInjectorServices()

@davidfowl that's a discussion for a different thread.

OK, but what if I need access to HttpContext inside my class library. I'll have to oblige all the developers using it to register HttpContextAccessor? My library is for web applications, so it should be able to get the request domain.
P.S. Passing HttpContext from client code's contorollers to my class isn't an option either: clients can create web applications without MVC.

You don't normally need to use IHttpContextAccessor as DI will inject the HttpContext for you if you ask for it; and then is scope is more controlled (as you shouldn't be using it outside of a request)

@benaadams, injecting HttpContext directly into a component's constructor is a bad idea, because HttpContext is runtime data and runtime data should not be injected into application components during construction.
It's for this very reason that Identity Framework fixed a bug prevented the SignInManager from taking HttpContext as constructor argument.

P.S. Passing HttpContext from client code's contorollers to my class isn't an option either: clients can create web applications without MVC.

How do client's call your library? HttpContext is not part of MVC, it's part of the underlying ASP.NET Core framework.

Sorry, my library needs request domain just to check its license (license keys are bound to domains). Its functionality doesn't depend on ASP.NET.

@Metamorphus add an extension method for your product that people call in ConfigureServices.
public static class MyProductServiceCollectionExtensions
{
    public static void AddMyProduct(this IServiceCollection services)
    {
        services.TryAddSingleton<IHttpContextAccessor, HttpContextAccessor>();
    }
}

@dotnetjunkie it's not really registered by default. It depends on what template you use... 😄

@davidfowl Can you give more information about this? What do you mean by template?

Its likely application insights adding it for their own usage. There's a new feature in the WebHost that lets various hosting environments inject application logic. Visual Studio injects the application insights logic in debug only and that's likely what you're seeing.

I can confirm that when ran outside of Visual Studio, the IHttpContextAccessor registration is missing, while when running from Visual Studio, it actually is registered.
Could you be more specific and describe what exactly is adding the IHttpContextAccessor registration?
I do think that it is a pretty bad idea to have such central abstraction registered conditionally. This can easily lead developers dazzled, since the application runs on their machine, but breaks once deployed.

A GitHub "stack trace"... 😂

https://github.com/aspnet/AzureIntegration/blob/9b9bb4dde7e9fa1bcfb31019ba9453882e1a318f/src/Microsoft.AspNetCore.ApplicationInsights.HostingStartup/ApplicationInsightsStartupLoader.cs#L32
https://github.com/Microsoft/ApplicationInsights-aspnetcore/blob/f696dd44f55055fad1774154219acf6c1e32c99b/src/Microsoft.ApplicationInsights.AspNetCore/Extensions/ApplicationInsightsWebHostBuilderExtensions.cs#L22
https://github.com/Microsoft/ApplicationInsights-aspnetcore/blob/c7e34ea2484f90a542e000c7a98656bf45ccf436/src/Microsoft.ApplicationInsights.AspNetCore/Extensions/ApplicationInsightsExtensions.cs#L130

ApplicationInsightsHostingStartup will be injected, discovered (notice the assembly-level HostingStartup attribute) and run from Visual Studio, Azure App Service etc., without you calling anything. Pretty scary, right? 😝


Configure with no-buildtest-c++ fails on Windows 10 with 1.1.1i

Configure command used (both from cmd and powershell):
perl ..\Configure no-buildtest-c++ no-tests no-external-tests --release VC-WIN64A-masm
Output:
Using os-specific seed configuration

It looks like you don't have either nmake.exe or dmake.exe on your PATH,
so you will not be able to execute the commands from a Makefile.  You can
install dmake.exe with the Perl Package Manager by running:

    ppm install dmake

Creating configdata.pm
Creating makefile

**********************************************************************
***                                                                ***
***   OpenSSL has been successfully configured                     ***
***                                                                ***
***   If you encounter a problem while building, please open an    ***
***   issue on GitHub <https://github.com/openssl/openssl/issues>  ***
***   and include the output from the following command:           ***
***                                                                ***
***       perl configdata.pm --dump                                ***
***                                                                ***
***   (If you are new to OpenSSL, you might want to consult the    ***
***   'Troubleshooting' section in the INSTALL file first)         ***
***                                                                ***
**********************************************************************

Ignore the "nmake.exe or dmake.exe" error that is reported from perl module IPC::Cmd calling IPC::Cmd::can_run('ranlib') since it does not cause a problem and is an ActiveState Perl issue.
Now when I run:
perl .\configdata.pm --dump
I get the following (reduced output here full output at bottom):
Command line (with current working directory = .):
    C:\Perl64\bin\perl.exe ..\Configure no-buildtest-c++ no-tests no-external-tests --release VC-WIN64A-masm

Perl information:
    C:\Perl64\bin\perl.exe
    5.26.1 for MSWin32-x64-multi-thread

Enabled features:
    buildtest-c\+\+

Disabled features:
    afalgeng                [not-linux] OPENSSL_NO_AFALGENG
    asan                    [default]   OPENSSL_NO_ASAN
    crypto-mdebug           [default]   OPENSSL_NO_CRYPTO_MDEBUG
    crypto-mdebug-backtrace [default]   OPENSSL_NO_CRYPTO_MDEBUG_BACKTRACE
    devcryptoeng            [default]   OPENSSL_NO_DEVCRYPTOENG
    ec_nistp_64_gcc_128     [default]   OPENSSL_NO_EC_NISTP_64_GCC_128
    egd                     [default]   OPENSSL_NO_EGD
    external-tests          [option]    OPENSSL_NO_EXTERNAL_TESTS
    fuzz-libfuzzer          [default]   OPENSSL_NO_FUZZ_LIBFUZZER
    fuzz-afl                [default]   OPENSSL_NO_FUZZ_AFL
    heartbeats              [default]   OPENSSL_NO_HEARTBEATS
    md2                     [default]   OPENSSL_NO_MD2 (skip crypto\md2)
    msan                    [default]   OPENSSL_NO_MSAN
    rc5                     [default]   OPENSSL_NO_RC5 (skip crypto\rc5)
    sctp                    [default]   OPENSSL_NO_SCTP
    ssl-trace               [default]   OPENSSL_NO_SSL_TRACE
    tests                   [option]    OPENSSL_NO_TESTS
    ubsan                   [default]   OPENSSL_NO_UBSAN
    unit-test               [default]   OPENSSL_NO_UNIT_TEST
    weak-ssl-ciphers        [default]   OPENSSL_NO_WEAK_SSL_CIPHERS
    zlib                    [default]
    zlib-dynamic            [default]
    ssl3                    [default]   OPENSSL_NO_SSL3
    ssl3-method             [default]   OPENSSL_NO_SSL3_METHOD

Please note that when I run nmake.exe it compiles files from .\test\buildtest_*.c. I have tried to modify configdata.pm manually in hopes that I could set it straight but that didn't work.
I am trying to speed up the build by configuring "no-buildtest-c++ no-tests no-external-tests".
Version Info:
Win 10 Ent 20H2
Perl 5.26.1
VS Ent 2019 16.8.3
OpenSSL 1.1.1i
Thanks,
SLDR
(Stephen L. De Rudder)
Full output:
Command line (with current working directory = .):

    C:\Perl64\bin\perl.exe ..\Configure no-buildtest-c++ no-tests no-external-tests --release VC-WIN64A-masm

Perl information:

    C:\Perl64\bin\perl.exe
    5.26.1 for MSWin32-x64-multi-thread

Enabled features:

    aria
    asm
    async
    autoalginit
    autoerrinit
    autoload-config
    bf
    blake2
    buildtest-c\+\+
    camellia
    capieng
    cast
    chacha
    cmac
    cms
    comp
    ct
    deprecated
    des
    dgram
    dh
    dsa
    dso
    dtls
    dynamic-engine
    ec
    ec2m
    ecdh
    ecdsa
    engine
    err
    filenames
    gost
    hw(-.+)?
    idea
    makedepend
    md4
    mdc2
    multiblock
    nextprotoneg
    pinshared
    ocb
    ocsp
    pic
    poly1305
    posix-io
    psk
    rc2
    rc4
    rdrand
    rfc3779
    rmd160
    scrypt
    seed
    shared
    siphash
    sm2
    sm3
    sm4
    sock
    srp
    srtp
    sse2
    ssl
    static-engine
    stdio
    threads
    tls
    ts
    ui-console
    whirlpool
    tls1
    tls1-method
    tls1_1
    tls1_1-method
    tls1_2
    tls1_2-method
    tls1_3
    dtls1
    dtls1-method
    dtls1_2
    dtls1_2-method

Disabled features:

    afalgeng                [not-linux] OPENSSL_NO_AFALGENG
    asan                    [default]   OPENSSL_NO_ASAN
    crypto-mdebug           [default]   OPENSSL_NO_CRYPTO_MDEBUG
    crypto-mdebug-backtrace [default]   OPENSSL_NO_CRYPTO_MDEBUG_BACKTRACE
    devcryptoeng            [default]   OPENSSL_NO_DEVCRYPTOENG
    ec_nistp_64_gcc_128     [default]   OPENSSL_NO_EC_NISTP_64_GCC_128
    egd                     [default]   OPENSSL_NO_EGD
    external-tests          [option]    OPENSSL_NO_EXTERNAL_TESTS
    fuzz-libfuzzer          [default]   OPENSSL_NO_FUZZ_LIBFUZZER
    fuzz-afl                [default]   OPENSSL_NO_FUZZ_AFL
    heartbeats              [default]   OPENSSL_NO_HEARTBEATS
    md2                     [default]   OPENSSL_NO_MD2 (skip crypto\md2)
    msan                    [default]   OPENSSL_NO_MSAN
    rc5                     [default]   OPENSSL_NO_RC5 (skip crypto\rc5)
    sctp                    [default]   OPENSSL_NO_SCTP
    ssl-trace               [default]   OPENSSL_NO_SSL_TRACE
    tests                   [option]    OPENSSL_NO_TESTS
    ubsan                   [default]   OPENSSL_NO_UBSAN
    unit-test               [default]   OPENSSL_NO_UNIT_TEST
    weak-ssl-ciphers        [default]   OPENSSL_NO_WEAK_SSL_CIPHERS
    zlib                    [default]
    zlib-dynamic            [default]
    ssl3                    [default]   OPENSSL_NO_SSL3
    ssl3-method             [default]   OPENSSL_NO_SSL3_METHOD

Config target attributes:

    AR => "lib",
    ARFLAGS => "/nologo",
    AS => "ml64",
    ASFLAGS => "/nologo /Zi",
    CC => "cl",
    CFLAGS => "/W3 /wd4090 /nologo /O2",
    CPP => "\$(CC) /EP /C",
    HASHBANGPERL => "/usr/bin/env perl",
    LD => "link",
    LDFLAGS => "/nologo /debug",
    MT => "mt",
    MTFLAGS => "-nologo",
    RANLIB => "CODE(0x31273e0)",
    RC => "rc",
    aes_asm_src => "aes_core.c aes_cbc.c vpaes-x86_64.s aesni-x86_64.s aesni-sha1-x86_64.s aesni-sha256-x86_64.s aesni-mb-x86_64.s",
    aes_obj => "aes_core.o aes_cbc.o vpaes-x86_64.o aesni-x86_64.o aesni-sha1-x86_64.o aesni-sha256-x86_64.o aesni-mb-x86_64.o",
    apps_aux_src => "win32_init.c",
    apps_init_src => "../ms/applink.c",
    apps_obj => "win32_init.o",
    aroutflag => "/out:",
    asflags => "/c /Cp /Cx",
    asoutflag => "/Fo",
    bf_asm_src => "bf_enc.c",
    bf_obj => "bf_enc.o",
    bin_cflags => "/Zi /Fdapp.pdb",
    bin_lflags => "/subsystem:console /opt:ref",
    bn_asm_src => "bn_asm.c x86_64-mont.s x86_64-mont5.s x86_64-gf2m.s rsaz_exp.c rsaz-x86_64.s rsaz-avx2.s",
    bn_obj => "bn_asm.o x86_64-mont.o x86_64-mont5.o x86_64-gf2m.o rsaz_exp.o rsaz-x86_64.o rsaz-avx2.o",
    bn_ops => "EXPORT_VAR_AS_FN SIXTY_FOUR_BIT",
    build_file => "makefile",
    build_scheme => [ "unified", "windows", "VC-common" ],
    cast_asm_src => "c_enc.c",
    cast_obj => "c_enc.o",
    cflags => "/Gs0 /GF /Gy /MD",
    chacha_asm_src => "chacha-x86_64.s",
    chacha_obj => "chacha-x86_64.o",
    cmll_asm_src => "cmll-x86_64.s cmll_misc.c",
    cmll_obj => "cmll-x86_64.o cmll_misc.o",
    coutflag => "/Fo",
    cppflags => "",
    cpuid_asm_src => "x86_64cpuid.s",
    cpuid_obj => "x86_64cpuid.o",
    defines => [ "OPENSSL_SYS_WIN32", "WIN32_LEAN_AND_MEAN", "UNICODE", "_UNICODE", "_CRT_SECURE_NO_DEPRECATE", "_WINSOCK_DEPRECATED_NO_WARNINGS", "OPENSSL_USE_APPLINK" ],
    des_asm_src => "des_enc.c fcrypt_b.c",
    des_obj => "des_enc.o fcrypt_b.o",
    disable => [  ],
    dso_cflags => "/Zi /Fddso.pdb",
    dso_extension => ".dll",
    dso_scheme => "win32",
    ec_asm_src => "ecp_nistz256.c ecp_nistz256-x86_64.s x25519-x86_64.s",
    ec_obj => "ecp_nistz256.o ecp_nistz256-x86_64.o x25519-x86_64.o",
    enable => [  ],
    ex_libs => "ws2_32.lib gdi32.lib advapi32.lib crypt32.lib user32.lib",
    exe_extension => "",
    includes => [  ],
    keccak1600_asm_src => "keccak1600-x86_64.s",
    keccak1600_obj => "keccak1600-x86_64.o",
    ldoutflag => "/out:",
    lflags => "",
    lib_cflags => "/Zi /Fdossl_static.pdb",
    lib_cppflags => "",
    lib_defines => [ "L_ENDIAN" ],
    md5_asm_src => "md5-x86_64.s",
    md5_obj => "md5-x86_64.o",
    modes_asm_src => "ghash-x86_64.s aesni-gcm-x86_64.s",
    modes_obj => "ghash-x86_64.o aesni-gcm-x86_64.o",
    module_cflags => "",
    module_cxxflags => "",
    module_ldflags => "/dll",
    mtinflag => "-manifest ",
    mtoutflag => "-outputresource:",
    padlock_asm_src => "e_padlock-x86_64.s",
    padlock_obj => "e_padlock-x86_64.o",
    perlasm_scheme => "masm",
    poly1305_asm_src => "poly1305-x86_64.s",
    poly1305_obj => "poly1305-x86_64.o",
    rc4_asm_src => "rc4-x86_64.s rc4-md5-x86_64.s",
    rc4_obj => "rc4-x86_64.o rc4-md5-x86_64.o",
    rc5_asm_src => "rc5_enc.c",
    rc5_obj => "rc5_enc.o",
    rcoutflag => "/fo",
    rmd160_asm_src => "",
    rmd160_obj => "",
    sha1_asm_src => "sha1-x86_64.s sha256-x86_64.s sha512-x86_64.s sha1-mb-x86_64.s sha256-mb-x86_64.s",
    sha1_obj => "sha1-x86_64.o sha256-x86_64.o sha512-x86_64.o sha1-mb-x86_64.o sha256-mb-x86_64.o",
    shared_cflag => "",
    shared_defines => [  ],
    shared_extension => ".dll",
    shared_extension_simple => ".dll",
    shared_ldflag => "/dll",
    shared_rcflag => "",
    shared_target => "win-shared",
    sys_id => "WIN64A",
    thread_defines => [  ],
    thread_scheme => "winthreads",
    unistd => "<unistd.h>",
    uplink_aux_src => "../ms/uplink.c uplink-x86_64.s",
    uplink_obj => "../ms/uplink.o uplink-x86_64.o",
    wp_asm_src => "wp-x86_64.s",
    wp_obj => "wp-x86_64.o",

Recorded environment:

    AR =
    ARFLAGS =
    AS =
    ASFLAGS =
    BUILDFILE =
    CC =
    CFLAGS =
    CPP =
    CPPDEFINES =
    CPPFLAGS =
    CPPINCLUDES =
    CROSS_COMPILE =
    CXX =
    CXXFLAGS =
    HASHBANGPERL =
    LD =
    LDFLAGS =
    LDLIBS =
    MT =
    MTFLAGS =
    OPENSSL_LOCAL_CONFIG_DIR =
    PERL =
    RANLIB =
    RC =
    RCFLAGS =
    RM =
    WINDRES =
    __CNF_CFLAGS =
    __CNF_CPPDEFINES =
    __CNF_CPPFLAGS =
    __CNF_CPPINCLUDES =
    __CNF_CXXFLAGS =
    __CNF_LDFLAGS =
    __CNF_LDLIBS =

Makevars:

    AR              = lib
    ARFLAGS         = /nologo
    AS              = ml64
    ASFLAGS         = /nologo /Zi
    CC              = cl
    CFLAGS          = /W3 /wd4090 /nologo /O2
    CPP             = $(CC) /EP /C
    CPPDEFINES      =
    CPPFLAGS        =
    CPPINCLUDES     =
    CXXFLAGS        =
    HASHBANGPERL    = /usr/bin/env perl
    LD              = link
    LDFLAGS         = /nologo /debug
    LDLIBS          =
    MT              = mt
    MTFLAGS         = -nologo
    PERL            = C:\Perl64\bin\perl.exe
    RC              = rc
    RCFLAGS         =

NOTE: These variables only represent the configuration view.  The build file
template may have processed these variables further, please have a look at the
build file for more exact data:
    makefile

build file:

    makefile

build file templates:

    ..\Configurations\common0.tmpl
    ..\Configurations\windows-makefile.tmpl
    ..\Configurations\common.tmpl

I can reproduce this issue on Linux too.
Looks like #9016 didn't quite have the desired effect. It seems to mean that buildtest-c++ is now always enabled.
I think @levitte probably needs to look at this one - but he is on holiday this week

I can reproduce this issue on Linux too.
Looks like #9016 didn't quite have the desired effect. It seems to mean that buildtest-c++ is now always enabled.
I think @levitte probably needs to look at this one - but he is on holiday this week

Note - this doesn't seem to impact master (only 1.1.1)

Note - this doesn't seem to impact master (only 1.1.1)

@levitte

Hmmm...  is this just about appearances, as in what configdata.pm -d displays.
Something to be noted is that C++ buildtests must be actively chosen, the default is no-buildtest-c++, and you must configure with enable-buildtest-c++ to get them at all.

Fix in #14081

Hmmm... is this just about appearances, as in what configdata.pm -d displays.

No. Its about what actually gets built.

Something to be noted is that C++ buildtests must be actively chosen, the default is no-buildtest-c++, and you must configure with enable-buildtest-c++ to get them at all.

As I said above:

It seems to mean that buildtest-c++ is now always enabled.

So, on 1.1.1, when I configure like this:
$ ./config -d --strict-warnings

When I build, I get lots of lines like this:
gcc  -Iinclude -pthread -m64 -Wa,--noexecstack -Wall -O0 -g -DDEBUG_UNUSED -DPEDANTIC -pedantic -Wno-long-long -Wall -Wextra -Wno-unused-parameter -Wno-missing-field-initializers -Wswitch -Wsign-compare -Wshadow -Wformat -Wtype-limits -Wundef -Werror -Wmissing-prototypes -Wstrict-prototypes   -MMD -MF test/buildtest_aes.d.tmp -MT test/buildtest_aes.o -c -o test/buildtest_aes.o test/buildtest_aes.c

Note: this happens on Linux as well.

Note - this doesn't seem to impact master (only 1.1.1)

Strangely there seems to be something not quite right on master too. On master, when I configure as per above, "configdata" tells me that buildtest is disabled. And yet I still get lots of compile lines compiling the buildtest files as per in 1.1.1

gcc  -Iinclude -pthread -m64 -Wa,--noexecstack -Wall -O0 -g -DDEBUG_UNUSED -DPEDANTIC -pedantic -Wno-long-long -Wall -Wextra -Wno-unused-parameter -Wno-missing-field-initializers -Wswitch -Wsign-compare -Wshadow -Wformat -Wtype-limits -Wundef -Werror -Wmissing-prototypes -Wstrict-prototypes   -MMD -MF test/buildtest_aes.d.tmp -MT test/buildtest_aes.o -c -o test/buildtest_aes.o test/buildtest_aes.c

Please tell me how this is about no-buildtest-c++, that looks like a normal compilation in C to me

C != C++.  The C++ builds are done with g++ or clang++, not with gcc or clang.  Assuming that the latter would disappear because of no-buildtest-c++ is a red herring.

However, it's quite possible that `no-tests' doesn't work quite right.  That's a problem in test/build.info.  Should that be treated in this issue as well (I don't quite like multi-fasetted issues, but...)

However, it's quite possible that `no-tests' doesn't work quite right.

I had a look, and it's like I guessed, that this wasn't quite respected.  It was easy enough to fix (just moving an ENDIF down), so I ended up adding that fix in #14081 as well.  This is an issue in master as well, but the fix is wildly different.

The version 1.1.1 is not supported anymore. Closing. If this is still an issue with currently supported releases, please open a new issue.


Update README.md

Added note for Azure support, intents, memory operations (including soft/hard deletes)

lgtm


Unexpected token { in JSON

Just updated to 1.0.0 and got this error when running (Node v7.5.0):
❯ updtr
Looking for outdated modules... ⠲ undefined:2
{"type":"finished","data":978}
^

SyntaxError: Unexpected token { in JSON at position 922
    at JSON.parse (<anonymous>)
    at /usr/local/lib/node_modules/updtr/lib/run.js:122:25
    at ChildProcess.exithandler (child_process.js:202:7)
    at emitTwo (events.js:106:13)
    at ChildProcess.emit (events.js:192:7)
    at maybeClose (internal/child_process.js:890:16)
    at Socket.<anonymous> (internal/child_process.js:334:11)
    at emitOne (events.js:96:13)
    at Socket.emit (events.js:189:7)
    at Pipe._handle.close [as _onclose] (net.js:501:12)

Do you have a yarn.lock file in your project dir?

I do

● use: yarn

Looking for outdated modules...
> yarn outdated --json --flat
⠴

 ERROR  Error when trying to parse stdout from command 'yarn outdated --json --flat': Unexpected token { in JSON at position 222
    at Object.parse (native)
    at parse$1 (/Users/miamed/.config/yarn/global/node_modules/updtr/dist/index.js:1424:17)
    at Object.outdated (/Users/miamed/.config/yarn/global/node_modules/updtr/dist/index.js:1440:31)
    at getUpdateTasksFromStdout (/Users/miamed/.config/yarn/global/node_modules/updtr/dist/index.js:684:5)
    at _callee$ (/Users/miamed/.config/yarn/global/node_modules/updtr/dist/index.js:720:42)
    at tryCatch (/Users/miamed/.config/yarn/global/node_modules/regenerator-runtime/runtime.js:65:40)
    at GeneratorFunctionPrototype.invoke [as _invoke] (/Users/miamed/.config/yarn/global/node_modules/regenerator-runtime/runtime.js:299:22)
    at GeneratorFunctionPrototype.prototype.(anonymous function) [as throw] (/Users/miamed/.config/yarn/global/node_modules/regenerator-runtime/runtime.js:117:21)
    at step (/Users/miamed/.config/yarn/global/node_modules/babel-runtime/helpers/asyncToGenerator.js:17:30)
    at /Users/miamed/.config/yarn/global/node_modules/babel-runtime/helpers/asyncToGenerator.js:30:13

If anyone else is delayed by this, you can use https://github.com/fritx/jayin as a quick (but blind) workaround.
cat package.json |\
  jayin "Object.keys(x.dependencies).join(' ')" |\
  xargs echo |\
  xargs yarn add -S

Or for devDependencies:
cat package.json |\
  jayin "Object.keys(x.devDependencies).join(' ')" |\
  xargs echo |\
  xargs yarn add -D

YMMV.

Fixed with https://github.com/peerigon/updtr/pull/75, shipped with v3.0.0 🚀


[REQ] Termux Support & Guide!

Termux requirements and guidance for fellow Android users.
Running on Termux (Android 14)

https://github.com/user-attachments/assets/a6fb559a-fb0c-4207-ae69-00787fcf9c9e

Playwright won't work on termux(you might have to disable minigame, since it is only one key/day). But playwright works on nethunter(nano version is enough)

I am also running on termux, but on nethunter. Playwright won't work on termux(you might have to disable minigame, since it is only one key/day). But playwright works on nethunter(nano version is enough)

Can you create a fork supporting Termux?

I am also running on termux, but on nethunter. Playwright won't work on termux(you might have to disable minigame, since it is only one key/day). But playwright works on nethunter(nano version is enough)

Can you create a fork supporting Termux?

I did it once(I even wrote one liner installation script). They keep on updating the bot every week. I  don't have time(sorry, I am too lazy for that) to manage. So i deleted the repo.
The changes in termux are we have to install rust for pydantic, disable minigame(in .env) enable comment for import playwright. That's it.
If anyone want to run the bot as per in orginal README, just install kali nethunter rootless nano version.
Brother, if you can maintain, you do it.

I am also running on termux, but on nethunter. Playwright won't work on termux(you might have to disable minigame, since it is only one key/day). But playwright works on nethunter(nano version is enough)

Can you create a fork supporting Termux?

I did it once(I even wrote one liner installation script). They keep on updating the bot every week. I don't have time(sorry, I am too lazy for that) to manage. So i deleted the repo.
The changes in termux are we have to install rust for pydantic, disable minigame(in .env) enable comment for import playwright. That's it.
If anyone want to run the bot as per in orginal README, just install kali nethunter rootless nano version.
Brother, if you can maintain, you do it.

Deleting a repo not an good idea, what if someone get an idea or caught any minimal mistake from other's sources.
Anyways thanx for the advice, I will try to do it, if got lot of free time.

Made a fork specially for Termux users.
https://github.com/DineshValor/HamsterValor

Made a fork specially for Termux users.
https://github.com/DineshValor/HamsterValor
Edit:- i have migrated this to new one, search in my repo, if still not found that's mean deleted or private due to some reasons.

Did you delete the repository?

Yep, bro currently I'm not active much so I can maintain it with @shamhi
repo. So I decided to delete it.
On Wed, 28 Aug, 2024, 19:01 Bogdan, @.***> wrote:

Made a fork specially for Termux users.
https://github.com/DineshValor/HamsterValor
Edit:- i have migrated this to new one, search in my repo, if still not
found that's mean deleted or private due to some reasons.
Did you delete the repository?
—
Reply to this email directly, view it on GitHub
https://github.com/shamhi/HamsterKombatBot/issues/2248#issuecomment-2315327476,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AXQTME35P3BAAP3XK4HOZ6TZTXGKBAVCNFSM6AAAAABM53PLO2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMJVGMZDONBXGY
.
You are receiving this because you modified the open/close state.Message
ID: @.***>


Using Structs (working fork)

First of, Moya looks great. At least most of it does. But the one single enum bugs me a bit. You have to define a single API endpoint in multiple parts of the same file. When the API grows, the enum gets really big since it is hard to divide it into multiple files.
I've made some changes in my fork of Moya, and perhaps this idea can be discussed here and maybe (when the idea is liked) it can be integrated in Moya itself. But it would break existing code using enums in the way I've implemented it currently.
The benefits of using structs are:

You can easily divide your API definition accros multiple files
All information regarding one single API Endpoint is contained within one place

Also I've made some extra changes which makes using the structs even nicer.

Automatic generating of the parameters in the request based on the structure of the struct using reflection (optional, you can still define your own parameters if you choose so)
Easy support for urls with formatting like: /user/{id}

I've got a full README.md on the Github page with some example code. I'd love to hear all of your opinions on it.
The biggest change in regard to the original version of Moya is the addition of the Smoya.swift file. Which is short for StructuredMoya. In there some protocols and extensions have been defined.
Next to that most of Moya remains the same. But there are some changes regarding generics to hook up the right enum. This has been removed and it accepts any TargetType now.
Also the baseURL has been removed from the TargetType and you must add that in your own endpoint. (The default endpoint has been removed). Since defining the baseURL in every struct seemed odd to me.
Again, I'd love to get some feedback. And am curious if people like this approach.

Hey, this is pretty cool! Thanks for the kind words, too.
I'll have more time this weekend to look closer at it. I agree there are limitations in centralizing things, especially as APIs grow. I've broken things into multiple enums in a project, and we've been talking about structs for a little bit. The use of the reflection API is really clever.
I'm not sure if we can "turn the ship" at this point (the structure of the enums is... pretty engrained into Moya), but I'd love to borrow some of the ideas from this. We should then consider linking to it from our docs, if you're interested in maintaining it as a separate project.

Yeah, I can understand that it would be quite a breaking change to move from the current enum structure to using structs. Nice to hear that you like the idea. Love to hear what you have to say when you've taken a close look.
And yeah, feel free to borrow whatever you like. Maybe a separate project is an idea. But maybe we can think of something else.

Hey! Sorry I didn't get to this over the weekend :grimacing: Work is very busy right now and I'm preparing for a few conferences.
@Moya/contributors Would anyone be free to take a look, give feedback, and open issues to discuss ideas we should consider adopting?

Instead of removing DefaultEndpointMapping, you could make easier to port by having it accept the baseURL, and then it would return a closure:
-public final class func DefaultEndpointMapping(target: Target) -> Endpoint<Target> {
-    let url = target.baseURL.URLByAppendingPathComponent(target.path).absoluteString
-    return Endpoint(URL: url, sampleResponseClosure: {.NetworkResponse(200, target.sampleData)}, method: target.method, parameters: target.parameters)
-}
+public final class func DefaultEndpointMapping(baseURL: NSURL) -> (target: Target) -> Endpoint<Target> {
+    return { target in
+        let url = baseURL.URLByAppendingPathComponent(target.path).absoluteString
+        return Endpoint(URL: url, sampleResponseClosure: {.NetworkResponse(200, target.sampleData)}, method: target.method, parameters: target.parameters)
+    }
+}

I like that fork's changes are pretty small - but important question: does the current enum-based system still work?
I know you're critical of it, but... well, you're an army of ~2 at the moment.  As long as we stay compatible with using enums to store the endpoints, I think this is great!  What I mean is: it's great to support both, but if we're just going to support one or the other, I vote that we stick with the enum.

It looks neat! I also thought about structures, but I really like enums' type safety. I'll take a closer look today/tomorrow if we could use something from it @ashfurrow. Also, great job @Matthijn 💣

@colinta That is a nice approach to the baseUrl. But since it is already part of the enum itself that would be kinda duplicate if you use the enum.
Also, In my current implementation the enum does not work anymore. Since with the enum you use generics to determine which enum cases can be passed. E.g: let service = new MoyaService<MyServiceEnum>(); and then service.request(.somethingInTheEnum()).
In my Struct based solution there is no generic MoyaService. You just create an instance like: let service = MoyaService(); and then you can pass any TargetType into the .request() method.

You could make a "StructMoyaService" which is not generic and an "EnumMoyaService" which is. Where they both inherit from (a non generic) "MoyaService" which will contain the bulk of the code which can be shared (which is most). But I don't think that is the way you want to go.
But I can't think of any way from the top of my head to support both in an elegant way.

I'm going to +1 this idea. I love Moya but while using it, one of my biggest frustrations was when working with/adding an endpoint... I would have to jump around these very large sections to get to the different parts of the enums. I think this is really great to have it all contained in a single place.

Interesting discussion. I'd prefer enums still for a couple of reasons:

I like how enums give my endpoints structure (e.g. AuthTarget.SignIn and ContentTarget.LatestPosts)
While the enum code has more boilerplate, it has less magic and so less of an understanding curve which is important to me in an open source library

That said I really like a couple of points in the structs (non generic providers especially) and would be interested in seeing how we could get some of them into Moya

@watson You could of course use container structs to group certain targets together if you so choose.

that's true, but it's more work for my use case which at the moment is a few different services with a few endpoints each (i.e. a micro services backend)
i might be biased though, I do love swift enums :)

I think I would love to see this kind of implementation too. For me personally, it makes it possible to fix my issue #383.
To make it more flexible, I mis an option to add other parameters to the struct which will not be automatically included in the query because of the reflection implementation.

@AvdLee If you don't add the ReflectiveParameters protocol extension to your Struct you can of course still define your own parameters variable within your struct, fine tuning on how you would like it to behave for that request.

Or maybe you meant that you would like to have some properties within the struct for other uses than generating the parameters then something like a parameterKeys could be introduced. That way you will have something like:
struct Foo: TargetType, ReflectiveParameters
{
    // The required properties for TargetType
    let path = "/foo"
    let method = Moya.Method.GET
    let sampleData = NSData();

    // Defines which properties must be added to the parameters through reflection
    let parameterKeys = ["baz", "bar"] // And for example when this is [ ] (default value) all properties will be added.

    // Will be added to the parameters
    var baz: String
    var bar: String

    // Will never be added
    var bak: Int
}

Instead of whitelisting blacklisting could be a possibility as wel.

Can't wait for something like this to be added to Moya, or to see the project switch to something like this in a new major release. The monolith enum and the scattered stuff never made sense to me either, it just felt... 'bad' and that's a sham, because Moya is actually really nice and tries to solve ugly code. I'd love to group relevant (self-containing) endpoint/target structs into .swift source files.
The use of Swift reflection is very interesting as well.
subscribes to notifications for this issue . :shipit:

I agree this is really interesting, I just want to provide a word of caution about radically changing Moya, and a work of assurance to our existing users who might be worried about large breaking changes.
I'm still unsure of the best approach to take here, but whatever we decide it will be done with careful consideration :+1:

Isn't it possible to just wrap your struct in an enum value, and forward the TargetType calls to your struct?
struct UserResource {
  let method: Moya.Method = .GET
  // ... and so on ...
}

enum API {
  case User(UserResource)

  var path: String {
    switch self {
      case let User(resource):  return res.path.resolve()
    }
  }
}

You could probably take this a step further:
struct UserResource: Moya.TargetType {
 // ...
}

enum API {
  case Resource(Moya.TargetType)
}

This lets people who use enums (which is still everyone at the moment) to continue without major refactors, but also lets you use your struct in the way you've proposed.

@colinta that is super cool. I think bringing that down deeper into Moya to have first-party support would be great, but at the very least that's a great solution for now and we should probably document this idea :100: :100: :100:
The first-party support could look something like a middleware between the Moya Provider type and Target type. Something that generates an Endpoint (which really doesn't need to be a generic at all). I'm just spit-balling, but having a middle layer that lets users use either sounds like an ideal world.
And I think it's worth noting that enums are a great solution – that's why we used them in the first place. It just happens that they don't scale well. On the other hand, structs do scale well, but spread domain knowledge of how an API works across several files (that's good for experts but can overwhelm beginners). Each approach has its advantages, so while an ideal world where either works is hard to imagine (and implement!) I think it may be worth striving for.

:+1:

Its been interesting watching this discussion. As @colinta and I work on the same project that uses Moya I haven't commented because we share a similar love for enums and I share his perspective. The possibility of the project ditching them concerns me though so I figured I better speak up.
The enum pattern introduced to us by Moya very early on has helped guide a lot of our decisions in our large and mature swift app. We love using enums for just about everything we can. They express a large (and complete) set of information in a very concise package.
However, we ran into the same issue that @ashfurrow mentions, they don't scale very well when there are a lot of endpoints. We haven't had a great solve until now. I like @colinta's struct in an enum suggestion. Its something that works within the existing Moya paradigm and may help solve the scale issue. I see a refactor to ElloAPI in the near future. :metal:

I adopted Moya for a new project back in November, and quickly found enums to not scale very well, also.
I came up with a solution similar to what @colinta and others have shared, with a struct wrapping a single (internal) TargetType adopter. I also extend the struct as a SignalTypeProducer, with magnificent results!
Here is an example taken from my playground file:
XCPlaygroundPage.currentPage.needsIndefiniteExecution = true

// http://jsonplaceholder.typicode.com
CurrentAPIHost = APIHost(serverContext: .PlaceholderTest)
JSONRequest("users")
    .startWithNext {
        print($0)
}

// https://api.guildwars2.com/v2
CurrentAPIHost = APIHost(serverContext: .GuildWars2)
JSONRequest("colors", parameters: ["ids":"all"])
    .startWithNext {
        print($0)
}

`JSONRequest` looks like this:
```swift
public func JSONRequest(endpointPath: EndpointPath, method: ReactiveMoya.Method = .GET, parameters: [String:AnyObject]? = nil) -> JSONTarget {
    let endpoint = JSONEndpoint.Request(endpointPath: endpointPath, method: method, parameters: parameters)
    return JSONTarget(endpoint: endpoint)
}
```

I think a face-to-face chat (over the internet) would be helpful here, to pin down:

What we want.
How to implement it.
Who will implement it.
Migration requirements.

I've scheduled a poll here for Thursday, March 10th: http://doodle.com/poll/ef4kbz3h8zi3crmz Sorry it can't be earlier, but I'm travelling until then. The times there are in EST, so go ahead and let me know when we can help this public hangout.

OK, going to have a hangout tomorrow at 2pm EST. I'll post a Google Hangout link here.

I forgot to reply to the poll, but I'll be there! :-D

OK, I've set up a hangout here: https://hangouts.google.com/hangouts/_/artsymail.com/moyahangout
Thought about recording it, but decided not to so people can speak freely. I'll summarize our discussion afterward and post here.

Thanks for continuing to encourage such a community-based project, @ashfurrow!  Looking forward to see where this goes.

I had a scheduling conflict, and so I appreciate the summary, @ashfurrow.
Having missed my opportunity to participate, I understand things are in motion; however, I'd still like to share my struct solution (KevinVitale/Shelley). It's promising to read so much of what motivated me reflected in the meeting notes, and as such, I'm looking forward to seeing what comes of it.
Thanks for such a great project, all.

I think we've addressed this in #427, pending some documentation updates in #428 and #429. I'm going to close this issue, but if anyone has any further feedback, please feel free to comment where you think is appropriate; we can re-open this issue or open new ones, so don't be shy!


Discussion: Type identity

Inspired by typing system in F# (I believe it's the same as in Haskell). I like the idea to express invariants and intention using types. It makes code explicit.
For example:
type ADiscriminatedUnionType =
    | Alternative1 of int * string
    | Alternative2 of bool
    | Alternative3

//somewhere else
let f du =
    // pattern matching
    match du with
    | Alternative1(x,s) -> printfn "x is %d, s is %s" x s
    | Alternative2(b) -> printfn "b is %A" b
    | Alternative3 -> printfn "three"

SUM supported by dry-types looks similarly, but it has one flow - you can't tell what type this specific object is.
CustomerID = Types::Strict::String
PostID = Types::Strict::String
some_id = CustomerID['abc']
# no way to tell either some_id is CustomerID or PostID

Even more problem, technically CustomerID and PostID are the same objects
CustomerID.object_id == PostID.object_id # true

I believe, that ruby is flexible and powerful enough, to make this possible within dry-types.
Let's discuss the best way to implement it.
My option:
We can define singleton method for every object that were passed via types constructor (e.g. CustomerID['abc']), which will return original type it was created with.
Then we can check:
case some_id.original_type
when CustomerID
...
when PostID
...
end

To workaround the same object_id problem, the most easier way is to use #dup
CustomerID = Types::Strict::String.dup
PostID = Types::Strict::String.dup

One more concern that should be implemented within scope of this issue is
post_id = PostID['123']
CustomerID[post_id] # exception, wrong type

Currently, with a regular string - it passes without any problem.
And one more concern: should we treat derived types as identity as well
ConstrainedPostID = PostID.constrained(min_size: 5)
post_id = ConstrainedPostID['abcdef']
# is post_id === PostID ?

Types can be distinguished with meta if needed. In fact, type equality is crucial for many applications of dry-types so this definitely won't change anytime soon. There is a related proposal in https://github.com/dry-rb/dry-types/issues/92 which seems to be more promising and allows to distinguish either types and values in a more elegant and safe way (keep in mind you can't have a singleton class for integers, symbols, and frozen objects).


Replace lazy_static with OnceLock

Codecov Report
Patch coverage: 100.00% and project coverage change: +0.01 :tada:

Comparison is base (d05859d) 68.21% compared to head (5378590) 68.22%.

:exclamation: Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #460      +/-   ##
==========================================
+ Coverage   68.21%   68.22%   +0.01%
==========================================
  Files          21       21
  Lines        4058     4060       +2
==========================================
+ Hits         2768     2770       +2
  Misses       1290     1290

Impacted Files
Coverage Δ

src/peer.rs
47.94% <ø> (ø)

src/ingest/tests.rs
91.66% <100.00%> (+0.14%)
:arrow_up:

src/publish/tests.rs
98.26% <100.00%> (+<0.01%)
:arrow_up:

src/server.rs
89.36% <100.00%> (ø)

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.