3.1.1-SNAPSHOT - CacheMissException: Object 'id=xxx' has no field named '__typename' when querying cache with addTypename.set("ifAbstract")

Summary
I wanted to try out changes introduced as a fix for: https://github.com/apollographql/apollo-kotlin/issues/3774 with the new addTypename.set("ifAbstract") option. I was still getting com.apollographql.apollo3.exception.CacheMissException: Object 'id=xxx' has no field named '__typename' so I tried to come up with a repro.
schema.sdl
type Viewer {
  shelf(id: ID!): Shelf!
  nodes(ids: [ID!]!): [Node]!
}

type Shelf {
  id: ID!
  books: [Book!]!
}

type Book implements Node {
  id: ID!
  name: String!
}

interface Node {
  id: ID!
}

queries
query Shelf($id: ID!) {
  viewer {
    shelf(id: $id) {
      id
      books {
        ...bookFragment
      }
    }
  }
}

query BooksByIds($ids: [ID!]!) {
  viewer {
    nodes(ids: $ids) {
      ...bookFragment
    }
  }
}

fragment bookFragment on Book {
  id,
  name
}

fails to fetch BooksByIds from cache, after pre-fetching it with Shelf query.
Am I wrong to expect ifAbstract to work in this case too? The test I added in the repro passes if I use the ifFragment option which I understood is considered as deprecated and is scheduled for removal.
Version
3.1.1-SNAPSHOT from ~yesterday.
Description
Here's full repro: https://github.com/mateuszkwiecinski/apollo-playground/commit/dda7a0d830392d178ef75147cf8ae60b6eeb4039

Thanks a lot for the reproducer 💜 . This is perfect timing as we wanted to make a release soon so we'll make sure this works before releasing.

So what's happening here is that __typename is not added in the first query but is required in the second. The transformed queries are like this:
query Shelf($id: ID!) {
  viewer {
    shelf(id: $id) {
      id
      books {
        # __typename is not added here because Book is a concrete, not-abstract type
        ...bookFragment
      }
    }
  }
}

query BooksByIds($ids: [ID!]!) {
  viewer {
    nodes(ids: $ids) {
      # __typename is added here because Node is abstract
      __typename
      ...bookFragment
    }
  }
}

I'm not sure there's a silver bullet here. I can see three possible solutions:
Solution1: "tryHard" 😅
This is the solution where we really want to minimize the number of extra __typename. We could have a global pass traversing all queries and remembering if a given type is queried through an interface (like is the case for Book through the Node interface here). If it's the case, we add __typename every time this field is queried.
Solution2: "ifAbstractOrHasSuperType" (naming TBC)
This is the intermediate solution. In addition to abstract types, we also add __typename unconditionally whenever the field type implements an interface or is part of an union to make sure we have a __typename if the field is queried through an interface (or union)
Solution3: "always"
Just add __typename always but that means we're back to step 1.
Thoughts
I'm not sure there's a silver bullet. "ifFragments" isn't great because of https://github.com/apollographql/apollo-kotlin/issues/3672#issuecomment-1007194022 so this one is certainly going to go away. For your specific case, I think 2. is the way to go (and certainly a good default).
We could go "tryHard" but TBH I don't think it's worth it. This not only would take some build time but would also make the transformed queries harder to understand.
I'll try to get solution2 in a SNAPSHOT so you can try it. Looks like you're not too wary of extra __typename, right? So it should be ok?

Thinking a bit more about this, I think I'm overthinking this 🙃 . Even "ifImplementsAbstractType" is not really bulletproof because it's not robust to schema changes that add an interface to an existing concrete type.
All in all, for 3.2.0 I don't think we need "ifImplementsAbstractType" so unless you have a strong objection, I'll certainly remove it. In the long run I think we should aim for these options:

"always": bullet proof option if using the cache
"ifAbstract": simple option if not using the cache
"ifPolymorphic": bare minimum of __typename

We'll keep "ifFragments" in the short term for compat reasons and remove it in the next major version.
In your specific case, that certainly means using "always" which is back to what 2.x is doing but I don't see an easy way out of adding __typename everywhere?

Thanks, Martin, for taking care of it almost immediately, and my apologies for not taking part in the discussion earlier 😬 Fortunately I don't have much to add 😛
If always is the only option which ensures valid cache data at a cost of adding __typename overhead to each type then that's OK for us :) That's going to be the same behaviour as in 2.x, so I think that's fine.
The only thing I'd like us to avoid is to explicitly add __typename to our queries - I'd be afraid we'd miss something, I don't think we have 100% of use cases covered with tests.
I also think these three choices you listed above would be enough in most cases, where one can pick performance vs idempotency to schema changes. The only idea not mentioned here (and I can think of right now) is to allow consumers to provide custom strategy when __typename gets added. I'm fairly sure we'd still go with always, but I can imagine someone wanting to optimize their queries and trying to remove __typename from types they know they won't ever have more than a single implementation, but yeah, that doesn't seem like a feature worth implementing right now.
I'll go ahead and close this issue. 3.2.0 introduced the always and it isn't scheduled for removal, so my original issue has been addressed, thanks!

Thanks! And apologies for rushing a bit the release. We've been sitting on a few important changes for some time and this release was direly needed. We can always do a patch release if you find anything during testing.
Regarding "always", I don't really see a way around it. It's slightly better than 2.x in that it saves __typename in inline fragments containing inline fragments:
{
  animal {
    ... on Pet {
       # 2.x adds __typename here, "always" doesn't, it only adds at the root of the "animal" field
       ... on Cat {
         meoww
       }
    }
  }
}

allow consumers to provide custom strategy when __typename gets added

Indeed, that's something I've been contemplating for a while but feeding Gradle with custom code is awkward. Basically, we'd need to have user-supplied code as an input to a Gradle task and this comes with a few challenges like how to know whether the logic has changed (for up-to-date checks) or exposing more APIs in the Gradle plugin which wouldn't get relocated. We're doing this for OperationOutputGenerator where the user has to manage up-to-date checks manually by bumping a version but I'm not a fan of this.
After playing with Gradle a bit, I think the idiomatic way to provide user-defined logic to Gradle tasks is to actually put the logic in a new Gradle task itself. In this case, the user could create a new AddTypename task that takes the .graphql files and copies them (with added __typename) to some other places in build/something and then wires the output to the codegen with srcDir(). It's some cost to setup but I don't really see a way around this. And it's all possible with the existing APIs actually. If you ever end up needing something like this, open a new issue and we can investigate.


Fix throwing exception on null parent.

I removed patch on master branch and created new patch (with same changes) to v5.x branch.

Thanks.


Having files ending in .gz stored next to the kineto trace may have unwanted side effects

Describe the Bug
In the current code base, the directory of the kineto trace is passed to HTA's TraceAnalysis class. This may have unwanted side effects if the directory contains other files besides the kineto trace that was passed to chakra_trace_link. For example, a zipped chakra execution trace (whose name will likely end in .gz) will crash the linker since HTA attempts to read the file ending in .gz, which fails since the chakra trace has a binary format.
Steps to Reproduce

Create a folder containing host_trace.json, device_trace.json and some random binary file ending in .gz (for example, a zipped fully converted chakra trace)
Run the trace linker on the host and device traces.

Expected Behavior

Only host_trace.json and device_trace.json are taken into consideration during linking

Screenshots

HTA, which received the directory device_trace.json was in attempts to read all present files, including the one ending in .gz and crashes.
Error log

Traceback (most recent call last):
  File "/home/.../PyCharmProjects/chakra/.venv/bin/chakra_trace_link", line 8, in <module>
    sys.exit(main())
             ~~~~^^
  File "/home/.../PyCharmProjects/chakra/.venv/lib/python3.13/site-packages/chakra/src/trace_link/trace_link.py", line 47, in main
    linker.link(args.rank, args.chakra_host_trace, args.chakra_device_trace, args.output_file)
    ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/.../PyCharmProjects/chakra/.venv/lib/python3.13/site-packages/chakra/src/trace_link/trace_linker.py", line 74, in link
    sync_deps = self.load_sync_dependencies(rank, chakra_device_trace)
  File "/home/.../PyCharmProjects/chakra/.venv/lib/python3.13/site-packages/chakra/src/trace_link/trace_linker.py", line 124, in load_sync_dependencies
    trace_analysis = TraceAnalysis(trace_dir=trace_dir)
  File "/home/.../PyCharmProjects/chakra/.venv/lib/python3.13/site-packages/hta/trace_analysis.py", line 36, in __init__
    self.t = Trace(trace_files, trace_dir)
             ~~~~~^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/.../PyCharmProjects/chakra/.venv/lib/python3.13/site-packages/hta/common/trace.py", line 330, in __init__
    trace_files if trace_files is not None else get_trace_files(self.trace_path)
                                                ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "/home/.../PyCharmProjects/chakra/.venv/lib/python3.13/site-packages/hta/common/trace_file.py", line 87, in get_trace_files
    ok, rank_to_trace_dict = create_rank_to_trace_dict(trace_path)
                             ~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^
  File "/home/.../PyCharmProjects/chakra/.venv/lib/python3.13/site-packages/hta/common/trace_file.py", line 48, in create_rank_to_trace_dict
    data = line.decode() if isinstance(line, bytes) else line

I believe this is fixed now via #167


[feature] Add support for USSD code execution

Is your feature request related to a problem? Please describe.
Currently, users are unable to run USSD codes directly from the app, limiting functionality for those in regions where USSD is a primary method for mobile services, notably in Africa.
For example, many users use USSD to check their balance, transfer funds or buy mobile airtime.
Describe your solution
I would like the application to natively support USSD code execution. This would enable users to trigger USSD operations via the Web APIs (such as checking an account balance, purchasing airtime or transferring funds). The application should send the USSD code automatically when the user triggers the desired action through a Web API, offering a seamless experience between the application and USSD interactions.
Additional context
This feature is particularly important for users in regions where mobile wallets, airtime purchases, and other mobile financial transactions are predominantly facilitated via USSD codes
I've already forked the project on GitHub and started implementing this feature. Throughout the process, I may need to contact you for advice and additional details to ensure the successful implementation of this feature.
@capcom6

I support this. 100%
On Tue, Sep 17, 2024, 5:50 PM Christian Alves Cyrille Amani <
@.***> wrote:

Is your feature request related to a problem? Please describe.
Currently, users are unable to run USSD codes directly from the app,
limiting functionality for those in regions where USSD is a primary method
for mobile services, notably in Africa.
For example, many users use USSD to check their balance, transfer funds or
buy mobile airtime.
Describe your solution
I would like the application to natively support USSD code execution. This
would enable users to trigger USSD operations via the Web APIs (such as
checking an account balance, purchasing airtime or transferring funds). The
application should send the USSD code automatically when the user triggers
the desired action through a Web API, offering a seamless experience
between the application and USSD interactions.
Additional context
This feature is particularly important for users in regions where mobile
wallets, airtime purchases, and other mobile financial transactions are
predominantly facilitated via USSD codes
I've already forked the project on GitHub and started implementing this
feature. Throughout the process, I may need to contact you for advice and
additional details to ensure the successful implementation of this feature.
@capcom6 https://github.com/capcom6
—
Reply to this email directly, view it on GitHub
https://github.com/capcom6/android-sms-gateway/issues/119, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/BD6R2AMFFILYVRQIPZ7ISUDZXBFVJAVCNFSM6AAAAABOLZCFYSVHI2DSMVQWIX3LMV43ASLTON2WKOZSGUZTCNJRGMYDAMQ
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

Can you clarify on this:
For I to send SMSs to my clients I use  the SMS-gateway app and   my sim
card account is debited.
My question is if you implement  the USSD feature, how will it work on the
part of the clients.  Let's say the client wants to send money using USSD
code via my web app.
Explain clearly if possible
On Tue, Sep 17, 2024, 5:53 PM Macson Maluwa @.***> wrote:

I support this. 100%
On Tue, Sep 17, 2024, 5:50 PM Christian Alves Cyrille Amani <
@.***> wrote:

Is your feature request related to a problem? Please describe.
Currently, users are unable to run USSD codes directly from the app,
limiting functionality for those in regions where USSD is a primary method
for mobile services, notably in Africa.
For example, many users use USSD to check their balance, transfer funds
or buy mobile airtime.
Describe your solution
I would like the application to natively support USSD code execution.
This would enable users to trigger USSD operations via the Web APIs (such
as checking an account balance, purchasing airtime or transferring funds).
The application should send the USSD code automatically when the user
triggers the desired action through a Web API, offering a seamless
experience between the application and USSD interactions.
Additional context
This feature is particularly important for users in regions where mobile
wallets, airtime purchases, and other mobile financial transactions are
predominantly facilitated via USSD codes
I've already forked the project on GitHub and started implementing this
feature. Throughout the process, I may need to contact you for advice and
additional details to ensure the successful implementation of this feature.
@capcom6 https://github.com/capcom6
—
Reply to this email directly, view it on GitHub
https://github.com/capcom6/android-sms-gateway/issues/119, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/BD6R2AMFFILYVRQIPZ7ISUDZXBFVJAVCNFSM6AAAAABOLZCFYSVHI2DSMVQWIX3LMV43ASLTON2WKOZSGUZTCNJRGMYDAMQ
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

@DevAlves1993
Thank you for your proposal.
The primary feature of the app is working with SMS, not USSD or calls. Adding support for USSD would significantly increase the complexity of the app and require changes on the backend side as well. Additionally, working with calls and USSD on Android is subject to additional restrictions from the OS.
I prefer the philosophy of "do one thing and do it well," similar to Linux tools. It would be great if you add this feature to your fork, but I don't have plans to incorporate it into the main app at this time.
I'm considering a separate project for calls and USSD functionality, but there's no ETA for this yet.

@Macson-Maluwa
USSD (Unstructured Supplementary Service Data) is not directly about sending money. Even if we implement a USSD feature in the app, it wouldn't enable receiving payments.
To enable actual money transfers or payments, you would need to integrate with a service provider or mobile operator that offers such functionality. This typically involves partnerships and adherence to financial regulations.

Thanks for the feedback!
On Wed, Sep 18, 2024, 4:06 AM Aleksandr @.***> wrote:

@Macson-Maluwa https://github.com/Macson-Maluwa
USSD (Unstructured Supplementary Service Data) is not directly about
sending money. Even if we implement a USSD feature in the app, it wouldn't
enable receiving payments.
To enable actual money transfers or payments, you would need to integrate
with a service provider or mobile operator that offers such functionality.
This typically involves partnerships and adherence to financial regulations.
—
Reply to this email directly, view it on GitHub
https://github.com/capcom6/android-sms-gateway/issues/119#issuecomment-2357344858,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/BD6R2AM3GGSPSZNUHJDEY73ZXDN4FAVCNFSM6AAAAABOLZCFYSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGNJXGM2DIOBVHA
.
You are receiving this because you were mentioned.Message ID:
@.***>

@capcom6
Thank you for your detailed response. I completely understand your point of view regarding the app's core focus on SMS and the potential complexity of integrating USSD support.
If possible, please keep us updated on this thread when you start working on the separate project for calls and USSD functionality. It would be great to follow along with your progress.
Thanks again!

@Macson-Maluwa
Yes, it is possible to perform actions through a sequence of USSD syntax. For example, to purchase a Data Pass, the following fictitious USSD code could be used:
#111*1*2*4*070707070707#.
This sequence replicates the entire USSD journey for buying a Data Pass. The actual syntax can vary depending on the country and mobile operator, but the principle remains the same.
In the case of money transfers via USSD, the client would follow a similar structured syntax to trigger the transfer. The USSD request would be executed directly by the client’s device, and their mobile account would be debited accordingly, similar to when using an SMS gateway. Again, the specifics would depend on the country and the operator involved.
This is just a simplified and fictitious example, but it demonstrates how USSD can be utilized to perform actions programmatically.


v3.0.0: allow naming of RabbitMQ client

This allows for things like event queues to be namespaced by the project's name (avoids collisions, for example).

closed by #41


Filter: operators for working with MAPs

          We have a similar opportunity for MAP types:

element_at: https://duckdb.org/docs/sql/functions/map#element_atmap-key
map_contains: https://duckdb.org/docs/sql/functions/map#map_containsmap-key
map_contains_key: https://duckdb.org/docs/sql/functions/map#map_contains_valuemap-value
map_contains_entry(map, key, value)](https://duckdb.org/docs/sql/functions/map#map_contains_entrymap-key-value)
Originally posted by @rpbouman in https://github.com/rpbouman/huey/issues/318#issuecomment-2558623836

Let's hold off on this untill the next DuckDB version drops. I noticed that the JSON reader has some changes which causes the blueworks dataset to return propery structs instead of MAPs, and also the MAP extraction methods have change so as to not return an array anymore. If we build this now we may find we have to redo it again.


ref: A better commit log format

I really don't want to port this abomination to Rust, let's simplify the commit log format
Two main changes:

uses float instead of weird datetime format we had before
avoids putting important information in headers. This was a hack to avoid changing the message format earlier, but it should be avoided. The timestamp is now in the payload of the message

The decode method is backward compatible and will only be removed in a future release.

On second thoughts, gonna move the encode change into a separate PR for deploy safety reasons


[Bug] It is possible to bypass word filters without triggering GIR

Word added

Proof of filter

Bypass

??


Structure support for write list by name

This is rebase. The original one can be found here: #213

The test still fails apparently because AdvancedHandler does not support structures at the moment. I'll see to that.

This PR also introduces sumup_read and sumup_write support on the AdvancedHandler of the testserver.

Pull Request Test Coverage Report for Build 686

11 of 13 (84.62%)  changed or added relevant lines in 2 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-0.1%) to 94.525%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

pyads/pyads_ex.py
5
7
71.43%

Totals

Change from base Build 669:
-0.1%

Covered Lines:
1174

Relevant Lines:
1242

💛  - Coveralls


Import master to stable-1.5

Import changes for STF 1.5.3

Fixes for 17.0 ir script (#380)
Move the SNMP trap delivery checks (#381)
Lock the bundle to OCP v4.10 (#385)
Make all certs 8yr expiry
Revert "Make all certs 8yr expiry"
Make all certs 8yr expiry (#387)
Expand support for OCP 4.11 (#391)
Force SGO checkout during build (#388)
Update oc to 4.11 in jenkins agent (#393)
Remove OperatorHub.io as a CatalogSource (#394)
Changes for 4.12 (#401)
Remove Loki from stf-run-ci (#405)
GHA checkout action v2 is deprecated (#407)
Implement SNMPtrap delivery controls (#404)
Implement changes for operator-sdk-1.26.0 testing (#411)
Expose ability to set certificate renewal target times (#406)
Fix default CA cert lifetime values in stf-run-ci (#414)
fix/client version mismatch (#415)
Allow oc client version override for Jenkins agent (#416)
Update CatalogSource for Prometheus Operator deployment (#419)
Fix path to right collectd service template file for OSP 13 (#418)
Latest oauth-proxy with bcrypt (#420)
[jenkins] Add custom context labels for github build status notification (#402)
Observability Operator for Prometheus & Alertmanager (#421)
Use stable-v1 cert-manager in CI for OCP >= 4.12 (#424)
Pinning prometheus version to ease migration to OBO (#425)
Fix auto-reversion to use_community (#427)
Fix duplicate alarm expressions (#426)
[zuul] Create zuul no-op job to test integration (#431)
Allow bundle deploy from unauth'd repo (#437)
Manage Operator dependencies with properties.yaml (#423)
Use the proper parameter for SNMP timeout (#435)
Support External Elasticsearch (#439)
Fix bug with supplied user.crt/key not persisting (#442)
Test with "external" elasticsearch (#441)
Revert cert-manager channel to "tech-preview" (#444)
Fix infinite reconcile loop (#445)
Add checks for not set sto and sgo bundle paths (#447)
Bump compatibility from 4.10-4.12 to 4.10-4.14 (#448)
Allow Grafana management in all observability modes (#449)
Add tests/infrared/17.1 (#440)
Corrected Hugepages alert summary to match severity (#451)
[stf-run-ci] Update vars in README (#459)
Update Dockerfile to create epel repo config when NO_PROXY is undefined (#458)
Add basic README for OSP 17.1 test deployment (#456)
Use fully qualified apiVersion on Routes (#453)
Clean up dependency installation in stf-run-ci (#455)
Fix static namespace reference (#464)
[ansible-lint] Use fqcn for modules (#457)
[stf-run-ci] Update pre-flight checks to be time-limited (#461)
Expose operator_sdk versions used in CI (#466)
Remove registry access config tags (#446)
generate_bundle: add logging (#463)
Improve logging (#471)
Relax pod admission controls when using a local catalog index (#450)
[stf-run-ci] Update pull_secret check for running bundles (#474)
Install ObO during index-based deployments (#472)
Update 17.1 helper scripts README (#476)
[issue#306] Add missing ClusterRoles (#465)
[stf-run-ci] Use {{ base_dir }} instead of relative paths (#460)
[stf-run-ci] Update base_dir in setup_stf_local_build.yml (#478)
[stf-run-ci] Add timeout to validate deployment (#477)
Adjust the default scrape interval (#454)
[stf-run-ci] Add requirements.txt (#484)
Add local generation difference check (#482)
Bump origin-ansible-operator base image (#487)
Remove logs API interface from STF (#483)
Reduce the Ceilometer pollsters in helper script (#489)
[zuul] Add base job for testing with zuul (#479)
Move the metrics_result retrieve (#493)
Increase sensubility smart-gateway ring buffer (#490)
Disable event pipeline management (#491)
[zuul] Use include_vars in prepare stage (#496)
Initial changes for QDR basicAuth (#481)
[zuul] use 'service-telemetry' as namespace (#497)
Add var to set QDR auth to none (#498)
[zuul] Add a job that does a local build and deploys an STF object (#480)
Bump requests from 2.27.1 to 2.31.0 in /build/stf-run-ci (#486)
Use vars for registry image container name (#452)
[stf-run-ci] Generate extra logs if preflight checks fail (#500)
Bump oauthlib from 3.2.0 to 3.2.2 in /build/stf-run-ci (#485)
[stf-run-ci][create_catalog] Swap query for a command task (#470)
Add requires infrastructure annotations (#499)
[stf-collect-logs] Add a role for log collection (#488)
[zuul] Add job to deploy from nightly bundles (#494)
Next release of STF will start at OCP 4.11 (#503)
Update base image to pass security scans (#502)
Cluster Observability Operator dependency (#504)
[better_logging] Show generate_bundle log when task fails (#501)
Don't deploy events SGs by default (#506)
Adjust CR annotation to matchdefault scrape interval
Fix bundle check test (#507)
Update path to internal TLS CA (#508)

@elfiesmelfie not sure if we'll need changes here to get this to pass/land?

The old "Linting" test was updated and renamed, so I'm skipping it (since it can't be run).


cannot pull images from docker registry

W0629 20:26:40.909717   16192 dockerimagelookup.go:220] Docker registry lookup failed: Get https://registry-1.docker.io/v2/: http: error connecting to proxy
https://<IP_ADDRESS>:8087: dial tcp <IP_ADDRESS>:8087: getsockopt: connection refused
error: no tags found on matching image stream: "openshift/nodejs"
The image stream "openshift/nodejs" exists, but it has no tags.
Use --allow-missing-imagestream-tags to use this image stream

This is an issue with your openshift cluster, not the sample app, please open an issue against origin and we'll try to help you there.


Handle container started as non-root better

description
It seems like a common issue that people try to start the container as non-root, which doesnt work with the ENTRYPOINT currently.
Users then seem to set ENVOY_UID and that only confuses the issue further.
I think we can do a couple of things.

print a very big warning in the entrypoint if it  detects that it has been started by a user other than root - perhaps with link to docs
allow the container to be started (with an explicit flag) as non-root in which case it wont try to drop perms or chown stuff etc

starting as non-root will probably work in most situations, esp if you ensure the env yourself - not sure off hand about stdout/err

this may be an issue for some specific frameworks or at least common setups - eg openshift

one idea i was thinking of for starting as non-root is that if the ENVOY_UID is set and the container is started with the same uid, perhaps it could skip the permissions dropping etc

@lizan any thoughts on this ?

/assign phlax

I think this is the only pattern where I see the user is created at the entrypoint. Should the envoy user be created as part of the container build, if it's always going to be envoy?

I forgot why we had chown in the first place, in most environment it isn't needed anyway... /dev/stdout is a symlink to /proc/self/fd/1

im trying to rem too - my vague recollection was that it was needed
i guess there are two options that might improve this - remove it (if unneeded), or add some logic to conditionally do whatever perms/user checking/dropping
i think the  main issue is that users are trying to start the container (from outside) as non-root, which can work i think, but wont with the current permission dropping

agreed, I'm ok with either way.

bump

bump

This is more important for us now that Envoy 1.14's xDS v2 API is deprecated. 1.14 was the last version we were able to run without encountering this issue.

@mtttcgcg there is a fix i think in #15115
ill follow up with an update to docs once that has landed

@ToniCipriani @mtttcgcg if you can confirm that this issue was resolved in #15115 ill add a note in docs and we can close this ticket i think

@ToniCipriani @mtttcgcg if you can confirm that this issue was resolved in #15115 ill add a note in docs and we can close this ticket i think

Looks good. Deployed envoyproxy/envoy-alpine-dev, container did not complain and I can see logs. Did a few basic tests and seems to work fine.
Thanks.

Also, will this be part of the next 1.17 release?

@Shikugawa would we be able to backport #15115 and/or include in current release ?

@phlax It makes sense. Affected versions are 1.15, 1.16 and 1.17, right?

great, thanks

Affected versions are 1.15, 1.16 and 1.17, right?

yep - just checked the v1.15-latest image and that has the problem entrypoint

@phlax We noticed the same problem again on one of our clusters. However it doesn't mention the additional message, and it tried to drop the permissions, resulting in the chown error again. I will have to check the service account configuration on what it's trying to run as.

im a little confused - im trying to figure out how either, it could be running as non-root but get a uid of 0, or, its running as root but cant chown


Refactors the Popup menus and GuiMenuBars

Refactors the Popup menus and GuiMenuBars to remove unneeded duplication and platform-specific/deprecated code.

Works for me.


feat: introduce class for reset list

Similar to clean-btn, this class is designed to reset the default browser styles, but for lists this time. It is more useful in Docusaurus code base than in Infima, because it makes sense to use it on infrequent elements, such as footer lists, so as not to clutter up the HTML markup too much.

👍  fine with that


Custom React components in Table rows?

I figured this would allow me to include my own custom components (in this case a customized input defined as TextField) in the Table, but I get this error and the rows do not render:
The only possible children of <Table> are <Thead>, <Tr>, or one <Tfoot>.

Is there no way to use this with other components at this point?

Yup, i have this same issue, it makes managing more complex code difficult.

How would you expect sorting to work for those elements?

the simplest way that comes to mind is getting the entire text content of the component and using that as a string value. if all values can be parsed as float / numeric, then sort / filter as numeric.

I had rows that had to hold array values and then since i didnt use the data prop sorting/filtering didnt work. Since my Tr wrapped my div i used the following for the sorting.
//We have to deconstruct the DIV array that we made
const arraySort = (a, b) => {
    const valuesA = a.props.children[0];
    const valuesB = b.props.children[0];

    if (valuesA.length === 0) {
        return 1;
    }
    if (valuesB.length === 0) {
        return -1;
    }

    //for some reason the array haves empty values with the "real" value
    const realA = valuesA[0].props.children.find(v => v !== ' ');
    const realB = valuesB[0].props.children.find(v => v !== ' ');

    return realA.localeCompare(realB);
};


fix: cmp popupmenu position

Currently, the popupmenu position of cmp is not aligned with input of smart-cmdline, because of its offset. Adjust the screen cursor position with offset should fix it. I try to access the offset directly by Cmdline.last().offset, but I'm not sure it's the right way.
Before

After

Hi! I think I may have fixed this over the last couple of days, since for me at least it looks correct without this PR.
Can you check again in your config?

Hi, I can reproduce this with this minimal init.lua:

init.lua

vim.cmd([[set runtimepath=$VIMRUNTIME]])
vim.cmd([[set packpath=/tmp/nvim/site]])
local package_root = "/tmp/nvim/site/pack"
local install_path = package_root .. "/packer/start/packer.nvim"
local function load_plugins()
  require("packer").startup({
    {
      "wbthomason/packer.nvim",
      {
        "folke/noice.nvim",
        requires = {
          "MunifTanjim/nui.nvim",
          "rcarriga/nvim-notify",
        },
      },
      {
        "folke/tokyonight.nvim",
      },
      {
        "hrsh7th/nvim-cmp",
        requires = {
          "hrsh7th/cmp-cmdline",
        },
      }
      -- ADD PLUGINS THAT ARE _NECESSARY_ FOR REPRODUCING THE ISSUE
    },
    config = {
      package_root = package_root,
      compile_path = install_path .. "/plugin/packer_compiled.lua",
      display = { non_interactive = true },
    },
  })
end
_G.load_config = function()
  vim.opt.termguicolors = true
  vim.cmd([[colorscheme tokyonight]])
  require("noice").setup()
  -- ADD INIT.LUA SETTINGS THAT ARE _NECESSARY_ FOR REPRODUCING THE ISSUE
  local cmp = require "cmp"
  cmp.setup {}
  cmp.setup.cmdline(":", {
    mapping = cmp.mapping.preset.cmdline(),
    sources = {
      { name = "path", max_item_count = 5 },
      { name = "cmdline", max_item_count = 10 },
    },
  })
end
if vim.fn.isdirectory(install_path) == 0 then
  print("Installing Noice and dependencies.")
  vim.fn.system({ "git", "clone", "--depth=1", "https://github.com/wbthomason/packer.nvim", install_path })
end
load_plugins()
require("packer").sync()
vim.cmd([[autocmd User PackerComplete ++once echo "Ready!" | lua load_config()]])

https://user-images.githubusercontent.com/40141251/201260016-1fe7d2c3-308e-4927-bc9c-1a9785b5c3d7.mov

Merged. Thank you!


curl dynamic flags after pod running

Introduction

Now dynamic flags will take effect after all pods phase is running, it takes too long and can't tolerate in some scenarios. We would like use a postStart hook to optimize it.
Contents

Related work

https://github.com/vesoft-inc/nebula-operator/pull/415


(BOLT-698) Add tests for output ordering

Ensure many lines our output in the right order. Acceptance test
currently expected to fail.

CLA signed by all contributors.

apply_winrm acceptance test is failing, working on it.

Needed to correctly quote stdin. All tests should now pass.

Original implementation was done in #67 and #110


Feature/flatpak

This pull request adds the Flathub as a platform to steam-buddy. Please do test this before merging.
It works quite well and installing and uninstalling applications is easy and seamless. The shortcuts file is created and the banners are downloaded. You can use it by clicking the flathub option on the platforms page. Let me know if you find any issues.
Things of note:

By default it only allows the installation of whitelisted applications. The whitelist can be found in flathub/whitelist.py
The flathub api is used. It is only used once on startup and when an application is opened in the web interface.
Images are loaded from the flathub website.
The description for an application is downloaded when it is needed.
Shortcuts are added during installation/uninstallation.
Installation/uninstallation happens in the background. Progress is tracked in the page of the application. This page will refresh every 5 seconds during installation to show progress if open.
Banners are never deleted. I didn't really see a need for it. The images are like 15 kb.
The flathub repo is added to the system when the application is started.
Which applications are installed is not tracked with shortcuts, but by asking flatpak.
The package python-requests is now a dependency of steam-buddy.

What I have tested so far:

Running this version of steam-buddy on Debian 10 and GamerOS 10. On GamerOS python-requests is required.
Installing applications
-Uninstalling applications

What was not tested:

Creating the shortcuts in Steam with steam-tweaks. I broke this on my system, it will take me some time to test this
Launching a shortcut

Known issues:

Updating flathub applications isn't implemented yet. It would probably be best to solve this with a service file.
The whitelist can only be updated for users by updating steam-buddy
We need a 404 page

Edits:

Updating has been implemented.

After some testing I've found that there are 2 issues when starting this patch set with the service file.

The service file limits which directories and files can be accessed, making all installations and uninstallations fail.
When the service is started initially, dns isn't available yet. This causes the applications to crash, because it can't reach the API.

I've added support for updating Flathub applications through steam-buddy. It now shows the current version and if the available version is different an update button will be shown.

Currently the service file is still stopping flatpak applications from being installed by limiting which directories can be accessed. I think the best solution for this would be to run the applications as the user instead so we don't need this limitation as much. What do you think @alkazar ?
All other major issues I've seen have now been fixed. Please let me know if you find any more.

I've changed the service file to allow flatpak installation. The code in this merge request should now be able to run on GamerOS like expected. The only change needed in the pkgbuild would the addition of flatpak and python-requests as dependencies.


Render annotation flags over track group headers

All track rendering was done on a big scrolling canvas element that sits behind the track dividing lines and shells. That is fine for rendering stuff in the track content area of the scrolling tracks, but the track content area of the headers of expanded groups does not scroll with the rest. They're not even a part of the same static layout, instead being positioned relatively and moreover sticking to the top of the panel as tracks scroll up beneath them. Because they are in a different layout to the canvas, they occlude anything painted on the canvas, so painting in the expanded group headers has no effect.
Consequently, it is necessary to give each expanded group header its own canvas on which we can draw content such as the stems of note flags.

Adding a note while the sticky open tack group is still there tends to make the annotations drawn on it stale.

Adding or removing a note while the sticky open track group is still there tends to make the annotations drawn on it stale.

Great find! Thanks. That was a tricky case and should be fixed in commit 4a6af55314e9f41d222254e2d27f5d70263bfb93.

While the annotation lines are no longer stale, the ones added while the sticky header is up are not the same length

While the annotation lines are no longer stale, the ones added while the sticky header is up are not the same length

Ah. What's happening here is that a group header has tucked up underneath another header that is not as high. In light mode, you would see that the header on top has a line ① running along its bottom edge and the apparent gap ② is just where the header underneath doesn't know it still needs to be drawn.

While the annotation lines are no longer stale, the ones added while the sticky header is up are not the same length

@ALevansSamsung this should be fixed with commit b450b905b677b68128418d9e0692744e58a959a1 which, overall, simplifies the code greatly by not trying to be clever about efficient drawing of something is known not to be complex to draw 😀

Still seems to be an issue with staleness
https://github.com/eclipsesource/perfetto/assets/121060410/41f7d461-7cb5-4413-9d4b-385108d611e1


Exception in serializer can cause channel handles to leak

Please answer these questions before submitting your issue.
Should this be an issue in the gRPC issue tracker?
yes
What version of gRPC and what language are you using?
1.11.0 CSharp
What operating system (Linux, Windows, …) and version?
Windows 10
What runtime / compiler are you using (e.g. python version or version of gcc)
.Net 4.6.1
What did you do?
If possible, provide a recipe for reproducing the error. Try being specific and include code snippets if helpful.
During a client unary async call the serializer threw an exception.  I then called ShutdownAsync on the channel.
What did you expect to see?
When the channel is shutdown it should do so cleanly
What did you see instead?
Warning message : "Channel shutdown was called but there are still 1 active calls for that channel"
Anything else we should know about your project / environment?
It looks like the immediate cause is that AsyncCall.UnaryCallAsync is calling "Initialize" which will artificially increase the active call count and also call DangerousAddRef on the channel handle.  When the serializer throws an exception the call is never started so there is no way for the code to ever get to the DangerousRelease call on the channel handle.  This could also happen if the INativeCall.StartUnary throws from the 'CheckOk' method.
I think the root cause of the issue is the manual ref counting that the code is trying to do.  It makes it likely that other problems like this will happen in the future ("GrpcEnvironment.RegisterChannel" is similar in that it could easily lead to channel leaks).  I assume the structure of the code was based on the C++ version where that manual reference counting is necessary. For .net it would be better to let the GC handle managing the reference counts.  The AsyncCall object already has a lifetime that encompasses the whole call, so if it held a reference to it's channel then that channel would stay in scope until the call completed.
If you agree I can submit a pull request for the changes.

@josiahdecker  can you provide a snippet that reproduces the problem?


feat(harbocluster): add owns in harborcluster controller

Signed-off-by: wangcanfeng<EMAIL_ADDRESS>

As we discussed offline yesterday, let's close this PR now.

As we discussed offline yesterday, let's close this PR now.


27 hydration storage

Created functions to save, delete, and retrieve data for the water screen via async storgae

Oops. Not ready to merge yet. Conflicts


Applies to Teams and Yammer

I believe the idea behind this section should be extended to apply to at least Teams and Yammer as well. While they may not be available on Personal O365 subscriptions, as far as I can tell, it is impossible to create a Cloud Resource rule that can differentiate between different organizations which generally creates the same risk. This is due to WIP network boundaries being limited to domains and subdomains, and not all O365 apps create unique subdomains for each tenant for all apps. E.g. you can make a rule to identify contoso.sharepoint.com and contoso-my.sharepoint.com as Contoso cloud resources, and will prevent workfiles from going to blahblah.sharepoint.com. Outlook, teams, yammer web apps cannot be protected though, because they all use the same domain and subdomains: outlook.office.com, teams.microsoft.com, yammer.com/contoso

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 5cf748a5-ebfd-ca94-0492-7580094618ee
Version Independent ID: b51e7663-6ef8-e1d3-6b68-7b897093c157
Content: Using Outlook on the web with WIP (Windows 10) - Microsoft 365 Security
Content Source: windows/security/information-protection/windows-information-protection/using-owa-with-wip.md
Product: w10
Technology: windows
GitHub Login: @DulceMontemayor
Microsoft Alias: dansimp

@ssccaavvookk thank you for reaching out and helping improve MS Docs. Are you suggesting adding a note to this article and have you checked to see that it does in fact apply to Teams and Yammer?

I suggest adding it, as of 3 months ago it did apply to Teams and Yammer, but I have taken down my WIP policies to switch to EndPoint DLP (which suffers from the same issue but isn't applicable here). I doubt it has changed, it's a bit of a fundamental implementation problem, but I can no longer verify.

@DulceMontemayor please can you confirm if this is still applicable and I'll do the PR for the article.

@ssccaavvookk Teams is considered a WIP-work only app, hence an unenlightened app.
In the scenario where you would use Teams only for corporate communication and data, you can add it to the Protected Apps section of your WIP policy. This will cause the app to enforce protection to all the data it works with or has access to.
I understand that the above behavior is not happening if you add Teams (or Yammer) to the Cloud Resource network element in your WIP policy (although i have not tested it), but why would you do that and not add Teams to the Protected Apps directly and thus have it treat everything as corporate data and have it protected?

@ssccaavvookk  Please let us know if you still have any question regarding the issue you have raised initially.
The issue is being closed since your question seems to be answered. Feel free to re-open this issue if you feel that it hasn't been answered or that there are further suggestions to improve the documentation itself.
Thanks.


RogueCFG

Hi, thank you for including me cfw to the list, but it needs a correction, it is not only for the RG350/M/P, it is also for the RG280V/M and RG300X too.

Dear  @Ninoh-FOX ,
thanks for pointing out this thing. I am going to update the list!


[SPIKE] Investigate optimal way to install/uninstall NGF for tests

This is a task to investigate most suitable approach to install and uninstall NGF in test framework, few of those options are:

Install and Uninstall via a helm command triggered using a golang fn or makefile.
Use k8s api to deploy/delete using a single manifest file generated from helm templates in one go.
Use individual functions to deploy each component i.e. Service, Secret, Deployment etc..

Results:
Specify the most optimal and reusable approach from the above 3 options.

Decided to use helm for this, allowing us the ability to configure various options when we need to.


How do we rotate Express-PouchDB's logs?

Express-PouchDB generates its own logs. We either need to figure out how to redirect the information into our logs or we need to figure out how to make sure the logs don't grow without bounds.

This is a security issue because an attacker could potentially figure out how to cause events that trigger a large number of log entries and so fill up the device.

would syslog support be an option?

That depends on whether they are only using express-pouchdb on the server or are using in Cordova as well. I don't think you can use syslog with Cordova.

Remember we are recording these logs locally on an Android or iOS device. So I doubt we'll have a syslog deamon. We probably just want some kind of simple file format (we are using Winston at the moment) and we need to make sure we rotate those logs so they don't fill up the device.

Gotcha, thanks!


Numpy 1.20.4

Hello,
First of all, thank you for your great code and useful database.
I found a strange thing in phonopy that I think is related to the numpy update. when I updated my numpy to 1.20.4.
When I used the following commands for the same files from the phonopy database the final results are different.
ph=phonopy.load("phonon.yaml", primitive_matrix='auto')
[freqs,evs] = ph.get_frequencies_with_eigenvectors(q=[0,0,0])
print(freqs)
Also, the differences are in the three first frequencies. The problem will be solved if used the NumPy with the lower version and maybe related to that in 1.20.4 passing shape=None to functions with a non-optional shape argument is deprecated.
Best regards

How much different?

for example for SRF2:
true results:[8.53421731e-08 1.02583202e-07 1.10882971e-07 6.50088989e+00 6.50088989e+00 6.50088989e+00 8.54769369e+00 8.54769369e+00 8.54769369e+00]
by numpy 1.20.4:
[-6.43583600e+00 -7.48237235e-01  1.44047595e-07  1.15390831e+00 3.87543142e+00  6.50088989e+00 6.50160434e+00  8.54769369e+00 8.54769450e+00]

Thanks for your report. I see. It's completely wrong.  But I have no means to reproduce it. I don't find numpy 1.20.4. So this issue will be kept for a while.

Do you still have the problem at the latest numpy?

Yes, I just avoid using Numpy 1.20

It seems 1.20.4 is not officially released, but newer versions are available. So I think this issue can be closed.


Add none country hint

Fixes: #9991
It also adds entries for change log including changes that happened in PR #10022

@chidozieononiwu  any idea why CI seems stuck here?

@chidozieononiwu Dozie Ononiwu FTE any idea why CI seems stuck here?

I really don't know it didn't even trigger the text Analytics pipeline. Perhaps closing and reopening or pushing another commit will fix this.


Bluetooth: host: Fix array overrun of client features bitset

Fix array overrun of client features bitset, where CF_BYTE_LAST was
incorrectly defined as 2 using module instead of divide.
Also fix additional bit-logic when number of bits would exceed a byte:

Inner for loop iterates from 0 to number of bits on a single byte.
Same bitmask used for all bytes.

Fixes: #32406
Signed-off-by: Joakim Andersson<EMAIL_ADDRESS>

Tested the function in isolation with a couple of different inputs and a longer byte array i.e more bits.
@Vudentz Do you know if this feature is covered by EDTT?


Allow module= attribute in complex enum variants

Allows use of the module attribute for complex enum variants (but not simple enums), like
#[pyclass(module = "custom_module"]
enum Complex {
    #[pyo3(module = "custom_module")]
    Variant(),
}

so that Complex.Variant.__module__ == "custom_module" instead of builtins
Useful to allow pickling (and introspection in general) for complex enums (as noted in https://github.com/PyO3/pyo3/issues/1517#issuecomment-806218369).

Thanks for the PR! I wonder, is there any case where users want the module of the variant to be different to the whole enum?
My feeling is no, in which case maybe it's better if we propagate the module from the enum onto all the variants...

@davidhewitt That was also my original feeling (at least, while the variants are still added to the same module as the whole enum). It is an even simpler PR then as well! latest commit has that change


deliver_scheduled_emails: Use a queue, instead of infinite retries.

deliver_scheduled_emails tries to deliver the email synchronously, and if it fails, it retries after 10 seconds.  Since it does not track retries, and always tries the earliest-scheduled-but-due message first, the worker will not make forward progress if there is a persistent failure with that message, and will retry indefinitely. This can result in excessive network or email delivery charges from the remote SMTP server.
Switch to delivering emails via a new queue worker.  The deliver_scheduled_emails job now serves only to pull deferred jobs out of the table once they are due, insert them into RabbitMQ, and then delete them.  This limits the potential for head-of-queue failures to failures inserting into RabbitMQ, which is more reasonable than failures speaking to a complex external system we do not control. Retries and any connections to the SMTP server are left to the RabbitMQ consumer.
We build a new RabbitMQ queue, rather than use the existing email_senders queue, because that queue is expected to be reasonably low-latency, for things like missed message notifications.  The send_future_email codepath which inserts into ScheduledEmails is also (ab)used to digest emails, which are extremely bursty in their frequency -- and a large burst could significantly delay emails behind it in the queue.
The new queue is explicitly only for messages which were not initiated by user actions (invitation reminders, digests, new account follow-ups) which are thus not latency-sensitive.

Fixes:

Screenshots and screen captures:

Self-review checklist

[ ] Self-reviewed the changes for clarity and maintainability
(variable names, code reuse, readability, etc.).

Communicate decisions, questions, and potential concerns.

[ ] Explains differences from previous plans (e.g., issue description).
[ ] Highlights technical choices and bugs encountered.
[ ] Calls out remaining decisions and concerns.
[ ] Automated tests verify logic where appropriate.

Individual commits are ready for review (see commit discipline).

[ ] Each commit is a coherent idea.
[ ] Commit message(s) explain reasoning and motivation for changes.

Completed manual review and testing of the following:

[ ] Visual appearance of the changes.
[ ] Responsiveness and internationalization.
[ ] Strings and tooltips.
[ ] End-to-end functionality of buttons, interactions and flows.
[ ] Corner cases, error conditions, and easily imagined bugs.

I think this might be a fine local improvement. I'd sorta been more thinking we should address this via improving the AbstractScheduledJob framework (well, it's not much of anything right now) towards being a queuing system that we can replace most (or all!) use of RabbitMQ for deferred work over time (with all the transactionality benefits that come with that). So for this detail, I'd consider extending AbstractScheduledJob with retry tracking logic that we think is reasonable.


Reenable po

/lgtm

This depends on https://github.com/openshift/cluster-olm-operator/pull/44 to merge

/retest

/retest

/retest

/approve


[Feature] Return attention_map for tokens

@zhuohan123 I have a use case where we utilize the attention map of the tokens in the prompt to decide whether to evict a section from the prompt if it’s running out of context length. It’s relatively easy to do with transformers and PyTorch but it’d be nice to get it from vllm too.
I can take a crack at implementing it too.

@WoosukKwon Where can I start if I tried to implement this?

I'm not sure whether this is actually doable with modern fast attention implementations (e.g., FlashAttention). These implementations avoid storing the full attention map to reduce the O(n^2) memory usage for the attention map. @WoosukKwon WDYT?

Oh right. Crap.
Umm, is it possible then to optionally disable flash attention in certain situations?

Closing this as out of scope


Rename YP_NODE_NODE to YP_NODE

As noticed by a few people the YP_NODE_ prefix felt redundant given it also ends in _NODE.

Don't merge until we update CRuby, otherwise this will break their CI.

Can we do this as a more proper migration so we don't have to worry about timing? Can we add a commit to alias YP_NODE to YP_NODE_NODE temporarily, then make the change in CRuby and then revert that single commit from here?

This feels very similar to the issue we had with TruffleRuby in YARP's CI.
Maybe importing YARP in CRuby shouldn't be done on every commit but manually or semi-automatically (example: a PR per week or so)?

Can we do this as a more proper migration so we don't have to worry about timing? Can we add a commit to alias YP_NODE to YP_NODE_NODE temporarily, then make the change in CRuby and then revert that single commit from here?

Fine by me, but please push that to this branch then.

I added the deprecated aliases in an extra commit, this should be safe to merge now.
I think it would be best to manually import yarp in CRuby for now, to avoid lots of ordering issues and tensions about what gets merged first, very similar to when TruffleRuby was in YARP's CI. It's just so much simpler and comfortable if adopting new APIs can be done when updating to a newer YARP commit (I have a fair amount of experience of that with YARP in TruffleRuby). I think it is likely there will be more breaking changes in YARP (breaking in terms of it would break CRuby if CRuby always pulls YARP master automatically).


Suggestion : Changing the looks of the buttons

Issue and Steps to Reproduce:
The current buttons of the card on the home page are too basic and doesn't match with the overall theme of the website and needs to be worked on.
Screenshots:

Can you assign this to me under gssoc?


What is the necessity of using mse loss instead of cross entropy?

Hi,
Thanks for the excellent paper and the generous share of this code base. I am trying to learn some details from this repo. Please allow me to ask a few questions:
Would you please explain why did you use mse loss rather than the normal cross entropy (or kl diversity loss) to train on the unlabeled data ? What is the philosophy behind this choice please?
By the way, I noticed other issues discussing about interleave the labeled and unlabeled batches. Do you think whether it would work if we first concatenate the labeled and unlabeled batch to one single big batch, and than run forward with this big batch. In this way, we could compute the labeled loss with the first batch size number of samples and compute the unlabeled loss with the rest samples?
What I mean is roughly like this:
inten = concat([x, u1, u2])
logits = model(inten)
loss_x = cross_entropy(logits[:batchsize], label_x)
loss_u = mse(logits[batchsize:], label_u)
loss = loss_x + lam * loss_u
...

Would you please tell me why you used the interleave method rather than the above method?

We use the Brier score (MSE loss) to make the training more robust to incorrect guesses. This is detailed in the paper on arxiv.
You could use a big batch. Just be mindful that with training=True the batch size must always be the same for batch_norm to behave properly. In particular, we run the model in guess_labels (on u1, u2) and we run it in the main body again (on x', u1', u2'). That's why we ended up running each group (x', u1, u2, u1', u2') separately since combining them would result in two mismatched batches.

In the settings of this code base, there are 5 forward computations per iteration: 2 for label guessing on u1 and u2, and 3 on the mixed input. All of these 5 forward computations would affect the running mean/var status of bn layers, though computation on u1 and u2 are not meant for training. Would it be better if we find a way to remove the impact on bn status brought by u1 and u2?

Actually only one forward computation affects the batch norm running mean and var. We explicitly make sure of that in the code by shuffling the batches contents (layers.interleave) and by ignoring the global updates (see post_ops) other than for that one batch.

Thanks, I will try it.

Just be mindful that with training=True the batch size must always be the same for batch_norm to behave properly.

Hi, thank you for your in-depth discussion. With regard to the statement above, I wonder is there any literature shows that it is best to ensure that the batch sizes are consistent for infering through the model during one batch iteration?

I am not sure about literature on the topic. It's just that with different batches sizes, I would expect the variance to change. So basically I erred on the safe side in the code. But feel free to experiment may it can be simplified.

Thanks for reply.

Hi, I have one more question about the CIFAR100 experiment.
In your paper, a 28-layer Wide Resnet model (which has 135 filters per layer, resulting in 26 million parameters) is used for CIFAR100. According to the tutorial in readme file, I just set the parameter filter equals to 135 and the code shows that total number of parameter is 25,985,290. I also notice that the filter number of the first conv layer(filter=16) is fixed in your ResNet class . so I wonder if this setup is the same as yours for getting the results in your paper. Thanks.

Yes it is the same setup. In reality we had a little bit less parameters than the methods we compared to, so yes the number is correct it's almost 26M.

Thanks

Hi,
Thanks for these replies, I am not familiar with tensorflow, so I feel maybe I better ask a bit more about the details :)
Did you use a batch of 64 interleaved samples to train the model with the other 128 samples neglected in computing the gradients of all parameters (or only use 64 samples to update bn, and use all the 192 samples to update other parameters)? If only 64 samples are used on all parameters, since I want to try a big batch of 192 samples per forward computation, should I multiply the learning rate by 3? As for the number of epoches and iterations, you trained with batch size of 64 for 1024 epoches and with each epoch consists of 1024 iterations, so if I train with 192 samples per iteration, I should reduce the number of iterations per epoch to 1024/3, am I correct ?

All batches (192 samples) are used in gradient computation.

Hello, I am very inspired to read your thesis, but when I run the code, I find that the code is missing files. Can you send me the complete code? Appreciate it!

I'm closing this issue since I answered the original question. For unrelated questions, please open new issues.


default perameters values

This isn't ideal, as it need to be handled as the way params are being passed and not in the code.

Find a way from existing structure only to handle the Exclude related parameter.
Closing this PR .


Tensor network simulator

Adds a basic tensor network simulator plugin to PennyLane. This plugin is still at an experimental stage.
Still to do (as future PRs):

add support for QubitStateVector and BasisState
include tests which rely on these in the test_tensornet.py file.
rethink shared tests between default.qubit and the tensor network backend
be more clever/efficient about tensor contractions
add ability for different backend executions (numpy, tensorflow, pytorch)

Also looks like you'll need to add tensornetwork to requirements.txt in order for it to be built on Travis :slightly_smiling_face:

Codecov Report

Merging #380 into master will decrease coverage by 0.1%.
The diff coverage is 98.93%.

@@            Coverage Diff             @@
##           master     #380      +/-   ##
==========================================
- Coverage   99.54%   99.44%   -0.11%
==========================================
  Files          32       33       +1
  Lines        2416     2510      +94
==========================================
+ Hits         2405     2496      +91
- Misses         11       14       +3

Impacted Files
Coverage Δ

pennylane/plugins/__init__.py
100% <100%> (ø)
:arrow_up:

pennylane/plugins/expt_tensornet.py
98.92% <98.92%> (ø)

pennylane/interfaces/tf.py
86.84% <0%> (-5.27%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1cfc126...0cb8ff3. Read the comment docs.

By the way, codefactor is complaining of trialing whitespace on expt_tensornet. Might as well black -l 100 the file now, since it's a new addition


Various cleanup of C++ client

And two new features:

Make Image movable without a copy.
Add an accessor to Context for media driver timeout.

A lot of this looks good. However removing the Context::conclude() changes the semantics for how contexts are used by convention. This will expand with archive and cluster being integrated.

Couple/Few things.

not sure about asserting on clz value of 0. I know why. Just not comfortable with it instead of a guard.
Context.conclude is needed in general. We could get away with it, but better to keep it.
I've tried various things with Images. Copying vs. Moving, etc. This is fine. But I think in general, we will eventually need to move to be compatible with how Java does it instead of trying to be local with the array. In other words, Image should be a standalone object. And the array in Subscription should be an array of pointers to Images.

@mjpt777 I have put back conclude() and just changed the initializers to defaults instead of nulls.
@tmontgomery What kind of guard do you have in mind instead of assertions? To me, it looked like passing zero to clz and ctz is a programming error that cannot be handled at runtime.
I suppose the standalone Image can also be moved, so the addition of move semantics here would not have to be retracted later.

I think a move for a reference doesn't make as much sense. But, in general, we can take this and then deal with that when it is needed.
The guard I was thinking of would be a return of the bit size as Java does.


change site_url to latest

Purpose

Describe the problems, issues, or needs driving this feature/fix and include links to related issues in the following format: Resolves issue1, issue2, etc.

Goals

Describe the solutions that this feature/fix will introduce to resolve the problems described above

Approach

Describe how you are implementing the solutions. Include an animated GIF or screenshot if the change affects the UI (email<EMAIL_ADDRESS>to review all UI text). Include a link to a Markdown file or Google doc if the feature write-up is too long to paste here.

User stories

Summary of user stories addressed by this change>

Release note

Brief description of the new feature or bug fix as it will appear in the release notes

Documentation

Link(s) to product documentation that addresses the changes of this PR. If no doc impact, enter “N/A” plus brief explanation of why there’s no doc impact

Training

Link to the PR for changes to the training content in https://github.com/wso2/WSO2-Training, if applicable

Certification

Type “Sent” when you have provided new/updated certification questions, plus four answers for each question (correct answer highlighted in bold), based on this change. Certification questions/answers should be sent to<EMAIL_ADDRESS>and NOT pasted in this PR. If there is no impact on certification exams, type “N/A” and explain why.

Marketing

Link to drafts of marketing content that will describe and promote this feature, including product page changes, technical articles, blog posts, videos, etc., if applicable

Automation tests

Unit tests

Code coverage information

Integration tests

Details about the test cases and coverage

Security checks

Followed secure coding standards in http://wso2.com/technical-reports/wso2-secure-engineering-guidelines? yes/no
Ran FindSecurityBugs plugin and verified report? yes/no
Confirmed that this PR doesn't commit any keys, passwords, tokens, usernames, or other secrets? yes/no

Samples

Provide high-level details about the samples related to this feature

Related PRs

List any other related PRs

Migrations (if applicable)

Describe migration steps and platforms on which migration has been tested

Test environment

List all JDK versions, operating systems, databases, and browser/versions on which this feature/fix was tested

Learning

Describe the research phase and any blog posts, patterns, libraries, or add-ons you used to solve the problem.

@shaniR , does the site_url: have to always remain as site_url: https://apim.docs.wso2.com/en/latest/ irrespective of whether the site is the latest doc space or not?

@shaniR , does the site_url: have to always remain as site_url: https://apim.docs.wso2.com/en/latest/ irrespective of whether the site is the latest doc space or not?

Yes,

The fixes for this PR is addressed in https://github.com/wso2/docs-apim/pull/1548


[ALLUXIO-3277] Increase worker timeout

https://alluxio.atlassian.net/browse/ALLUXIO-3277

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/20764/
Test PASSed.

Merged build finished. Test PASSed.

Yes, I will cherry-pick

https://github.com/Alluxio/alluxio/pull/7675


migration guide

Description
Add your description here, if it fixes a particular issue please provide a
link
to the issue.
Checklist

[ ] Commit sequence broadly makes sense and commits have useful messages.
[ ] New tests are added if needed and existing tests are updated.
[ ] Relevant logging and metrics added
[ ] CI passes. See note on CI.
[ ] Any changes are noted in the changelog.md for affected crate
[ ] Self-reviewed the diff

Note on CI
If your PR is from a fork, the necessary CI jobs won't trigger automatically for security reasons.
You will need to get someone with write privileges.  Please contact IOG Partner Chains developers to do this
for you.

IMO this PR should go to master. Otherwise it won't be visible at v1.4.0 tag when we make it.


Remember the directory path in registries

Bug reported by<EMAIL_ADDRESS>Description:
Remember the directory path when adding parts through  registries.

I've actually added this already.  Now, there is a separate importPath in Preferences.  This remembers the path for whenever you import anything (registries, parts, sequences).


Author and Creation time is "NA" on index page

Need to double check if the API is working as expected.

fixed.

verified.


BlazorHero Deploying Error 405

hi all i got this error after deploying
i read for it more than one week please help
6blazor.webassembly.js:1 info: Microsoft.AspNetCore.Authorization.DefaultAuthorizationService[2]
Authorization failed. These requirements were not met:
DenyAnonymousAuthorizationRequirement: Requires an authenticated user.
/signalRHub/negotiate?negotiateVersion=1:1
   Failed to load resource:/signalRHub/negotiate?negotiateVersion=1:1

the server responded with a status of 405 (Method Not Allowed)
blazor.webassembly.js:1 info: Microsoft.AspNetCore.Authorization.DefaultAuthorizationService[2]
Authorization failed. These requirements were not met:
DenyAnonymousAuthorizationRequirement: Requires an authenticated user.
blazor.webassembly.js:1
   crit: Microsoft.AspNetCore.Components.WebAssembly.Rendering.WebAssemblyRenderer[100]
  Unhandled exception rendering component: net_http_message_not_success_statuscode, 405, Method Not Allowed

System.Net.Http.HttpRequestException: net_http_message_not_success_statuscode, 405, Method Not Allowed
at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode()
at Microsoft.AspNetCore.Http.Connections.Client.HttpConnection.NegotiateAsync(Uri url, HttpClient httpClient, ILogger logger, CancellationToken cancellationToken)
at Microsoft.AspNetCore.Http.Connections.Client.HttpConnection.GetNegotiationResponseAsync(Uri uri, CancellationToken cancellationToken)
at Microsoft.AspNetCore.Http.Connections.Client.HttpConnection.SelectAndStartTransport(TransferFormat transferFormat, CancellationToken cancellationToken)
at Microsoft.AspNetCore.Http.Connections.Client.HttpConnection.StartAsyncCore(TransferFormat transferFormat, CancellationToken cancellationToken)
at System.Threading.Tasks.ForceAsyncAwaiter.GetResult()
at Microsoft.AspNetCore.Http.Connections.Client.HttpConnection.StartAsync(TransferFormat transferFormat, CancellationToken cancellationToken)
at Microsoft.AspNetCore.Http.Connections.Client.HttpConnectionFactory.ConnectAsync(EndPoint endPoint, CancellationToken cancellationToken)
at Microsoft.AspNetCore.Http.Connections.Client.HttpConnectionFactory.ConnectAsync(EndPoint endPoint, CancellationToken cancellationToken)
at Microsoft.AspNetCore.SignalR.Client.HubConnection.StartAsyncCore(CancellationToken cancellationToken)
at Microsoft.AspNetCore.SignalR.Client.HubConnection.StartAsyncInner(CancellationToken cancellationToken)
at System.Threading.Tasks.ForceAsyncAwaiter.GetResult()
at Microsoft.AspNetCore.SignalR.Client.HubConnection.StartAsync(CancellationToken cancellationToken)
at CAP.Client.Shared.MainLayout.OnInitializedAsync()
at Microsoft.AspNetCore.Components.ComponentBase.RunInitAndSetParametersAsync()
at Microsoft.AspNetCore.Components.RenderTree.Renderer.GetErrorHandledTask(Task taskToHandle)
d.printErr @ blazor.webassembly.js:1

Any help?

ERROR in  await hubConnection.StartAsync();
in MainLayout
and help

@arkannet Hi, did you figure this out?

Replace the following:
services.AddMediatR(Assembly.GetExecutingAssembly());

With:
services.AddMediatR(cfg=>cfg.RegisterServicesFromAssemblies(Assembly.GetExecutingAssembly()));


Support binary deployments of .NET 5 apps onto a Linux/Windows stack

Currently, the extension supports .NET Core app deployment from source via cf push. This feature should introduce the ability to deploy .NET Core apps to a CF running TAS Windows as described in these docs and described below:

.NET Core on Windows
To run .NET Core apps on the Windows stack, you must use the Binary buildpack. Follow the steps below to configure your Cloud Foundry manifest appropriately.

In the app manifest file, specify the windows stack and a custom start command. This file should be in the same folder as your .csproj file and be marked copy always.

applications:

name: MY-AWESOME-APP
stack: windows
command: cmd /c .\MY-AWESOME-APP --urls=http://<IP_ADDRESS>:%PORT%

> 2. Publish the project using the dotnet CLI or Visual Studio.
>
> `$ dotnet publish -o ./publish -r win10-x64 -f netcoreapp2.1`
>
> 3. A publish folder should now exist. Navigate to this publish folder and verify the manifest.yml file exists. Run cf push to deploy your app.
>
> ```
$ cd publish
> $ cf push MY-AWESOME-APP

The docs seem to recommend the following deployment strategies:

Type of .NET App
Buildpack

.NET Core pushed to Linux stack
.NET Core

.NET Core pushed to Windows stack
Binary

.NET Console (not recommended to run on CF; see footnote)
Binary

ASP.NET MVC
HWC

ASP.NET Web Forms
HWC

ASP.NET WebAPI Apps
HWC

Windows Communication Foundation (WCF)
HWC

Footnote:

Console apps are not recommended to run on Cloud Foundry because they do not normally run as a single instance and are not highly available. If you need to run a console app on Cloud Foundry, see Console Applications in the .NET Cookbook.

Closing; the strategy for this work has changed & is now tracked by #162


Set ogr2ogr auto-detect size limit to the whole file

Prior to this change, only the first 100,000 bytes of the input
file were scanned to detect field types. This led to incorrect
behavior when a dummy row was added at the end to force a numeric
field to treated as text.
This change removes the scan limit to resolve the problem at the
expense of potentially slowing down large imports.

We need to verify the degradation in performance with this change before rolling out the change in production..


searchChatsOnServer after creating new chat returns empty list

I'm attempting to search for a new chat created from createNewBasicGroupChat and createNewSupergroupChat via searchChatsOnServer. However, the new chats do not appear in the search results until after I restart tdlib--at which point the same query results in the correct chats being returned.
Couple other notes:

I see updateNewChat and other update events relating to the newly created chat
I have the following parameters:

			"@type":                    "tdlibParameters",
			"use_test_dc":              "0",
			"database_directory":       dbFolder,
			"files_directory":          dbFolder,
			"use_file_database":        true,
			"use_chat_info_database":   true,
			"use_message_database":     true,
			"use_secret_chats":         false,
			"api_id":                   env("TELEGRAM_API_ID", ""),
			"api_hash":                 env("TELEGRAM_API_HASH", ""),
			"system_language_code":     "en",
			"device_model":             "App",
			"system_version":           "0.1.0",
			"application_version":      "0.1.0",
			"enable_storage_optimizer": true,
			"ignore_file_names":        false,

I've tried calling getChatHistory and getChat after creating the new chat to see if that would change anything, but to no avail

Any help would be appreciated! Thanks!
TDLib Version: 1.5

Result of searchChatsOnServer is cached until client restart.
This is not a problem, because searchChatsOnServer is designed to complement searchChats and only add to its results chats unavailable locally. The just created chat can be successfully found by searchChats, so there is no practical reason for it to be returned by searchChatsOnServer.

Make sense, thanks!


if eq does not support empty strings

Describe The Bug
This is similar to #47 , but this is when the empty string is part of a conditional statement like if.
To Reproduce
This works:
e = eq "" ""
if ${e}
  # do stuff
end

This fails:
if eq "" ""
  # do stuff
end

Error Stack
ERROR - Error while running duckscript: Source: Unknown Line: 16 - Two arguments are required.

hmm seems it breaking the if/else parsing....

@wmmc88 this should be fixed now. thanks for reporting


[5.6] Fix params order in Broadcast Manager.

Fix intended to reorder the parameters in the Pusher Broadcaster to match the order expected by the latest Pusher constructor.

Could you reference what changes in the Pusher class you're referring to? I don't see the change in the Pusher class. https://github.com/pusher/pusher-http-php/blob/master/src/Pusher.php#L43

Laravel suggests Pusher ~3.0 which matches the constructor parameters in the BroadcastManager class.

This will unfortunately break for all existing users though.

Here's the code from BroadcastManager:
new Pusher($config['key'], $config['secret'],
            $config['app_id'], $config['options'] ?? [])

Here's the constructor from the latest Pusher:
public function __construct($auth_key, $secret, $app_id, $options = array(), $host = null, $port = null, $timeout = null)

The order is "key", "secret", "id", "options".
Links:
https://github.com/pusher/pusher-http-php/blob/master/src/Pusher.php#L43
https://github.com/laravel/framework/blob/5.6/src/Illuminate/Broadcasting/BroadcastManager.php#L213

This PR is incorrect.
The root cause of the incorrect Pusher installation is described in this closed issue thread: https://github.com/laravel/framework/issues/23224 - @jrojas28 if you look at that, you'll see how to solve your problem.
Although that thread raising as interesting question why composer is installation a different branch. Maybe @GrahamCampbell can advise in that thread as an expert in that type of stuff?

@laurencei your shot was correct. The issue was on my end due to lacking php-curl. After installing it and retrying it pulled from the proper branch & installed correctly.
I'm guessing the old branch is there & is pulled as back up because when lacking php-curl, Pusher relies on the HTTP Client features developed within that branch. Still, that's just a wild guess.
I'll leave this PR open for another hour or two in case @GrahamCampbell would like to provide some advices, but otherwise and as stated by @laurencei , this PR is wrong & will be closed.

@GrahamCampbell - can we tighten the pusher composer requirements, so the other branch is not pulled, and instead composer fails?


Expose more configuration options on the command line

Describe the Feature
It would be good to get more config options exposed on the command line. At the very least --case option to control keyword, function and data type case with values upper, lower.  This would allow one to use the command line without having to write multiple config files.
Why do you want this feature?
Usability.

I think it's a fairly sensible request.
I see a couple of possible ways to implement it:
Separate command-line option for each config option
sql-formatter --keyword-case=upper --tab-width=4

The benefit is that the config options would be in the most natural form for command-line usage.
The downside is a lot of new config options (more complex command-line API) and a lot of additional code to maintain.
Some generic command line option to specify any config option
sql-formatter --define keywordCase=upper --define tabWidth=4

This sort of solution would keep the API pretty simple. But it still requires a good chunk of code to deal with each option. Like it needs to know that 4 needs to be converted to a number, while upper has to remain a string.
A way to write the JSON config inline
sql-formatter --config '{"keywordCase": "upper", "tabWidth": 4 }'

This would be by far the simplest to implement and maintain. Though on the downside it's the worst to type out on the command line. But on the plus side there's no translation needed to convert config in JSON file to config on command line.
I personally like the last approach the best.

Implemented in 15.1.0 release.

Thank you for being so responsive and implementing this so quickly. Awesome work, and I like the approach taken as well. This is very flexible now.


Allow steps to catch exceptions for use in subsequent steps

E.g.
"When doing something"
	.When(() => foo.Bar())
	.Catch();

"Then something"
	.Then<Exception>(ex => ex.Message.Should().BeOfType<InvalidOperationException>());

}}}

or
"When doing something"
	.When(() => foo.Bar())
	.Catch<InvalidOperationException>();

"Then something"
	.Then<InvalidOperationException>(ex => ex.Message.Should().Be("bar"));

"When... "
   .x(c => c.Catch(() => SomethingBad());

"Then... "
   .x(c => c.Exceptions[0].Should().BeOfType<FooException>());

"When... "
   .x(c => c.Try(() => SomethingBad());

"Then... "
   .x(c => c.Catch().Should().BeOfType<FooException>());

Try pushes onto a stack, Catch pops off it.

Better to use xunit's Record.Exception() to capture the exception. Sharing variables between steps is a separate concern.


Error: NotLeaderForPartition when sending payload

Bug Report
I'm getting this error every time I try to send a payload. Unlike #245 , this happens in the Producer.
Environment

Node version: 12.3.1
Kafka-node version: 4.1.3
Kafka version: <IP_ADDRESS>

Include Sample Code to reproduce behavior.
The code is a little more complex, but the client and producer are created in this way and the function responsible for sending the payload is called correctly.
this.client = new kafka.KafkaClient();
this.producer = new kafka.Producer(this.client);

let payload = [{
      topic: 'Topic A',
      messages: JSON.stringify(dataObj),
    }];

this.producer.send(payload, function(err, data) {
        console.log("Error:", err);
      });

The logged error is the following one:
Error: Error: NotLeaderForPartition
    at Object.<anonymous> (node_modules/kafka-node/lib/protocol/protocol.js:952:17)
    at Object.self.tap (node_modules/binary/index.js:248:12)
    at Object.decodePartitions (node_modules/kafka-node/lib/protocol/protocol.js:950:95)
    at Object.self.loop (node_modules/binary/index.js:267:16)
    at Object.<anonymous> (node_modules/kafka-node/lib/protocol/protocol.js:165:8)
    at Object.self.loop (node_modules/binary/index.js:267:16)
    at decodeProduceV2Response (node_modules/kafka-node/lib/protocol/protocol.js:944:6)
    at KafkaClient.Client.invokeResponseCallback (node_modules/kafka-node/lib/baseClient.js:589:18)
    at KafkaClient.Client.handleReceivedData (node_modules/kafka-node/lib/baseClient.js:569:10)
    at KafkaClient.handleReceivedData (node_modules/kafka-node/lib/kafkaClient.js:1367:48)
    at Socket.<anonymous> (node_modules/kafka-node/lib/kafkaClient.js:797:10)
    at Socket.emit (events.js:200:13)
    at Socket.EventEmitter.emit (domain.js:471:20)
    at addChunk (_stream_readable.js:290:12)
    at readableAddChunk (_stream_readable.js:271:11)
    at Socket.Readable.push (_stream_readable.js:226:10)

Thanks for your contribution!

Removing the logs-dir made the trick. In my case, it was at /var/lib/kafka, but I read that other people have them at /tmp/kafka-logs


Add sentiment analysis feature

Calculate sentiment score for comments and store in datastore
Display sentiment scores alongside comments on index.html.

@megankj -- cc'ing you in!


Displaying geocoder results

Need better feedback if user is not seeing any results, actions that users can take to get results

Need smarter messages

if results are not empty; and if new results are empty; keep old results until new results are not empty

If nothing found, show suggestion to search by address, street, neightborhood, etc as links. Links go to specific geocoders search (e.g. address only search)

Going to open a new issues, specifically to address the way search results disappear/reappear while typing in search text

I'm gonna close this for now. I opened and closed issue 38 to address this. I'll reopen if needed after next round of user testing


#:set does not support metric functions #:set button_padding (dp(18), dp(12))

Importing doesn't enable this either
#:import dp kivy.metrics.dp
#:set button_padding (dp(18), dp(12))

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

This was fixed in https://github.com/kivy/kivy/issues/4061 and calling dp should also work without an import.
from kivy.app import App
from kivy.lang import Builder

kv = """
#:set label_text str(dp(12))

Label:
    text: str(label_text)
"""

class TestApp(App):
    def build(self):
        return Builder.load_string(kv)

if __name__ == '__main__':
    TestApp().run()


Add details for running notebooks in the readme

I had some troubles with the commands for running the notebooks, so I added some details for alternative ways.

I think this
https://github.com/openEOPlatform/openeo-pg-evalscript-converter/blob/10f20d094fbb52f07c22edda3b75cfdf71d5e012/README.md?plain=1#L197
should no longer be needed as it was added to Pipfile:
https://github.com/openEOPlatform/openeo-pg-evalscript-converter/blob/10f20d094fbb52f07c22edda3b75cfdf71d5e012/Pipfile#L17
So now just pipenv install --dev and pipenv lock --pre should suffice. Can you check just pipenv install --dev --pre?

Running only pipenv install --dev --pre, cd notebooks and pipenv run jupyter notebook works, so I updated the section, so it's shorter and cleaner


Trojan warning of 0.1.4 binary on Windows 10

After downloading  and executing the clone.exe binary, Windows Defender prevents it from running marking it
https://www.microsoft.com/en-us/wdsi/threats/malware-encyclopedia-description?name=Trojan%3AWin32%2FBearfoos.A!ml&threatid=2147731250

Trojan:Win32/Bearfoos.A!ml
Detected by Microsoft Defender Antivirus
Aliases: No associated aliases

Summary
Microsoft Defender Antivirus detects and removes this threat.
This threat can perform a number of actions of a malicious hacker's choice on your PC.

Mmh I have to investigate... Since I never heard anyone else reporting this and the binary is not build on my personal PC but via github actions, I'm pretty sure this is a false positive.
Are you getting this error with the 0.1.5 or with an older version, too? (I think 0.1.5 is not linked in the readme, but you can download it on the releases page).

just tested 0.1.5. it seems fine. The built-in Window 10 Defender did not complain.

just tested 0.1.5. it seems fine. The built-in Window 10 Defender did not complain.

Mmh ok... strange. I'll try to find out what's going on, but this one will be hard to reproduce.

Closed due to no further reported problems...


Implement lock mechanism

When using the keychains, it's possible to run commands indefinitely unless you lock your keychain.
I'm adding a sub-mechanism to drop the key in keychain so it forces to re-ask for the master password on the next command.
For the ones more concerned about using the keychain:

you have a configuration available to not store your master password in the keychain
you can create a cron job or a script that locks your vault at a given period of time or by an action

In my opinion, these lock mechanisms should rather be managed directly by the keychain, but I understand it is temporarily simpler like that, and easier to find and understand than dcli configure save-master-password false && dcli configure save-master-password true that has the same semantics.


Required cucumber-json-formatter no longer available for json reports

The documentation for generating cucumber reports states that the cucumber-json-formatter binary is expected to be installed.
I happen to have already downloaded this for MacOS, but when going back to download it for linux and integrate it into our pipeline, I see it no longer exists.  This issue says it was removed because "We don't need it anymore - Cucumbers are keeping their native JSON formatters."
Is there an alternative solution?  Preferably a nodejs package so we can include this in our package devDependencies rather than installing a binary.

Is there an alternative solution?

There is not.
It's code is kept alive for now, ref https://github.com/cucumber/common/pull/1916#issuecomment-1121458995 and the docs have been updated with the correct URL (37779508e463cdc6e6041476afd17b93447027eb).

Thanks for the link and updated docs!

I have create a wrapper due to same issue. You can give a try. @jwmickey
https://www.npmjs.com/package/@deepakvishwakarma/cucumber-json-formatter


Apply the permission level to the API as in the document said.

Is your feature request related to a problem? Please describe.
Currently, API only cares if the ID Token can be verified. But the API need to apply more permission level
Ex. Only the author or the tag subscriber of the assignment can get the assignment detail.

This was done in #35.

Implemented in #35


Per-entity drop pools for specific cards, packs, etc...

Is your feature request related to a problem? Please describe.
In relation to something I'd asked on the Discord here, it would be super nice if there was a way to set per-mob drop pools. Using the example I gave earlier, something like making it so that you can configure the plugin to have a 99% chance of dropping a zombie card from a zombie, with a 1% chance of dropping a random other card from the pool.
Describe the solution you'd like
Ideally, it'd be super awesome if there was a yml for dictating drop pools, likely along the lines of something like this(pseudocode, so will likely need some adjustments I'm sure if not a total rewrite):
Because github hates letting me format things, I've set it up on Pastebin over here.
Describe alternatives you've considered
The only way I can see doing this as possible is to use another plugin to add them to the drop pool which, while doable, strikes me a bit as just unnecessary bloat if this feature was to be added. So I thought I'd ask and, if the request was denied, I could then resort to that as a less graceful but still functional solution.
Additional context
No real extra context, other than the Discord convo I linked earlier. :)

I noticed in your example you referenced mythic mobs, this is already possible via an addon.
Although it may be outdated, feel free to test it and let me know if it works.
You can find the addon here: https://github.com/TreasureIslandMC/TCMythicMobs


Add support for exporting realm with member consent.

I came across a bug while testing this by exporting from a randomly created realm instead of default zulip realm.

Issue seems to be the user_profile_id of the message is the same value from the old realm. Trying to figure out the fix

@hackerkid this might have to do with the fact that development environment doesn't handle system bots correctly (they should not be in the zulip realm, but are).  To debug, figure out what user has that ID.

@timabbott Actually this bug is present in master branch and is not an issue with this PR. The issue is that we are not replacing data-user-id in mention messages.
"
@ron<\/span> Hello<\/p>

When the message gets exported to the new realm the  rendered_content of mention message would still have the data-user-id of the profile from the old realm which results this in error.

@hackerkid ahh OK, that's https://github.com/zulip/zulip/issues/11293.  Doing the high priority part of that PR's proposal would be pretty valuable.

@hackerkid this is nice work!  I especially liked the test setup work you did.  I made a bunch of changes documented above, and merged this as 06983298ba6b1ad5062e025a82437a14f2411bea.  A few notes on follow-ups:

The logic in export_partial_message_files related to PMs is pretty subtle; I think it'd be really valuable to expand the new test you added to include direct 1:1 PMs from a non-exported user to another, from a non-exported user to an exported user, and from an exported user to a non-exported user, and make sure the right thing happens for this, with similar asserts as to which of those message IDs were included in the export.  We should also make sure we test those 3 cases in the "full export" code path as well, since I can easily imagine that breaking.
This algorithm doesn't currently handle the issue of messages sent to private streams at times when none of the consenting users were subscribed (e.g. if it's a private stream with protected history and all the consenting users joined later).  I think for now we should just expand the new test to setup that corner case and have an assert that the message is exported (with a comment saying this behavior is wrong and we should fix it).  Probably the actual fix will involve doing some post-processing of the exported message files to remove any zerver_message elements with no associated zerver_usermessage elements in private streams with protected history (we'll probably need to do something annoying to pass in the recipient IDs for those).
I added a TODO comment about private stream metadata being exported, which is a related issue to the last bullet. I don't think it's worth taking any action on that issue at this time.
I think it is probably worth doing the lxml parsing work to fix data-user-id values in mentions in rendered_content in the import tool that you ran into here.

@hackerkid can you take care of those?
@whoodes FYI; once we have your "public export" feature merged, we'll want to add an option to do an "export with consent" based on this code path (and with a button that sends the announcement from notification-bot).  But definitely lower priority than finishing the main "public export" feature as it has a lot of extra complexity.


Stucking at Progress: 94

After '$ node watch.js' im stucking on PROGRESS: 94
LOG:
`
Starting Server on port 8000..
PROGRESS:94
events.js:125
throw er; // Unhandled 'error' event
^
Error: watch ENFILE
at exports._errnoException (util.js:734:11)
at FSWatcher.start (fs.js:1216:11)
at Object.fs.watch (fs.js:1242:11)
at createFsWatchInstance (/Users/programiro/Sites/website-relaunch-2016/styleguide/structure/_node-files/node_modules/chokidar/lib/nodefs-handler.js:37:15)
at setFsWatchListener (/Users/programiro/Sites/website-relaunch-2016/styleguide/structure/_node-files/node_modules/chokidar/lib/nodefs-handler.js:80:15)
at FSWatcher.NodeFsHandler._watchWithNodeFs (/Users/programiro/Sites/website-relaunch-2016/styleguide/structure/_node-files/node_modules/chokidar/lib/nodefs-handler.js:228:14)
at FSWatcher.NodeFsHandler._handleFile (/Users/programiro/Sites/website-relaunch-2016/styleguide/structure/_node-files/node_modules/chokidar/lib/nodefs-handler.js:255:21)
at FSWatcher. (/Users/programiro/Sites/website-relaunch-2016/styleguide/structure/_node-files/node_modules/chokidar/lib/nodefs-handler.js:473:21)
at FSReqWrap.oncomplete (fs.js:73:15)
`

Had to install latest Harp:
$ sudo npm install -g harp@next
https://github.com/sintaxi/harp/issues/564

Thank you for the reference @programiro !


How can I spend several utxo if one of them is P2tr?

I have succeeded spend multiple UTXO with different script types in one transaction, but if there is P2tr in them, it will fail.  "non-mandatory-script-verify-flag (Invalid Schnorr signature)
If possible, add some new Pt2r samples.
I want to add Multi-Sig functionality to my wallet to send and receive Bitcoin transactions
What I came up with, the user needs to create a P2wsh  address, to create this script, the user needs to add the public keys of the accounts they want to unlock the script with. The wallet generates a p2wsh address using the p2wsh script and public keys, the user gives the address of this script to the clients to receive the amount, and finally, the user can spend this script through the private key of the account. Is the method correct?
Is it possible to decode utxos script hash (p2wsh)  to find conditions? (how many signatures)
Thanks

Example https://github.com/karask/python-bitcoin-utils/blob/master/examples/spend_multi_input_p2tr_and_p2pkh.py demonstrates how to spend both taproot and legacy UTXOs in one tx.
Note that we add an empty witness that corresponds to the legacy utxo: tx.witnesses.append( TxWitnessInput([]) )

Example https://github.com/karask/python-bitcoin-utils/blob/master/examples/spend_multi_input_p2tr_and_p2pkh.py demonstrates how to spend both taproot and legacy UTXOs in one tx.
Note that we add an empty witness that corresponds to the legacy utxo: tx.witnesses.append( TxWitnessInput([]) )

I tried this method, and it worked great with P2PKH. However, when mixing it with P2WPKH or P2SH, I encountered the same error (RPCError: got code -26 with the message 'non-mandatory-script-verify-flag (Invalid Schnorr signature)'). Could you take a look at this code and help me identify the issue?
The code is ready for broadcasting
def main():
    # always remember to setup the network
    setup('testnet')
    priv1 = PrivateKey("cV3R88re3AZSBnWhBBNdiCKTfwpMKkYYjdiR13HQzsU7zoRNX7JL")
    priv2 = PrivateKey("cSfna7riKJdNU7skpRUx17WYANNsyHTA2FmuzLpFzpp37xpytgob")
    pub1 = priv1.get_public_key()
    pub2 = priv2.get_public_key()
    fromAddress1 = pub1.get_taproot_address()
    fromAddress2 = pub2.get_address()
    # p2tr utxo
    txid1 = 'd18b272e110fadd753dc30e08e9b12c1effe58a4f1253ae2289248b2994d0bf8'
    vout1 = 0
    # P2wpkh UTXO
    txid2 = 'd18b272e110fadd753dc30e08e9b12c1effe58a4f1253ae2289248b2994d0bf8'
    vout2 = 1
    amount1 = 100000
    amount2 = 100000
    amounts = [amount1, amount2]
    script_pubkey1 = fromAddress1.to_script_pub_key()
    script_pubkey2 = fromAddress2.to_script_pub_key()
    utxos_script_pubkeys = [script_pubkey1, script_pubkey2]

    toAddress = P2pkhAddress('mnv5EAfPMTbfatz5ZbT6uUKpK7E5MftBAv')
    txin1 = TxInput(txid1, vout1)
    txin2 = TxInput(txid2, vout2)
    txOut = TxOutput(to_satoshis(0.0000022), toAddress.to_script_pub_key())

    tx = Transaction([txin1, txin2], [txOut], has_segwit=True)
    sig1 = priv1.sign_taproot_input(tx, 0, utxos_script_pubkeys, amounts)
    sig2 = priv2.sign_segwit_input(tx, 1, script=script_pubkey2, amount=amount1)

    tx.witnesses.append(TxWitnessInput([sig1]))
    tx.witnesses.append(TxWitnessInput([sig2, pub2.to_hex()]))
    print("\nRaw signed transaction:\n" + tx.serialize())
    # broadcast fail !
    # got error RPCError: got code -26 with msg "non-mandatory-script-verify-flag (Invalid Schnorr signature)".
    #

I apologize for the numerous questions I am asking you, and I appreciate your responses. If feasible, could you please assist me in creating a test transaction with a memo or note? I am sincerely grateful for your help.

After conducting multiple tests without success, I reached the conclusion that the issue might reside with the node itself. It appears that the problem lies within the node. I managed to successfully send this transaction using a different node
TX ids
88558b495f28ff4799a4dacf40ee5cd1666850fed4847b8315256a5ae22527f8
02429ffe2f82f587d8e7a6e925201d3e7f219ac3af3688f2bc5c2770204e5d5c

However, none of these transactions have received confirmation. What do you think could be the reason for this?

Bitcoin testnet is known to be quite unstable. If you notice there aren't new blocks from yesterday:
https://live.blockcypher.com/btc-testnet/
That is why signet was created; to provide a more stable env for testing. I use both and although testnet is less reliable it does the job just fine.