[Feature] deploy workflow-step should support not filling properties

Is your feature request related to a problem? Please describe.
When there is no explicit policy to use, application workflow should support deploy workflow-step without properties.
Describe the solution you'd like
apiVersion: core.oam.dev/v1beta1
kind: Application
metadata:
  name: create-config
  namespace: default
spec:
  components:
  - name: nginx
    properties:
      image: nginx
      imagePullPolicy: IfNotPresent
    traits:
    - properties:
        port:
        - 80
      type: expose
    type: webservice
  workflow:
    steps:
    - name: write-config
      properties:
        config:
          key1: value1
          key2: 2
          key3: true
          key4:
            key5: value5
        name: test
      type: create-config
    - name: deploy-my-component
      type: deploy

It reports error:
    - firstExecuteTime: "2023-03-13T11:15:50Z"
      id: y6elrnvs43
      lastExecuteTime: "2023-03-13T11:15:50Z"
      name: write-config
      phase: succeeded
      type: create-config
    - firstExecuteTime: "2023-03-13T11:15:50Z"
      id: 4uuotv2ze2
      lastExecuteTime: "2023-03-13T11:16:42Z"
      message: 'run step(provider=multicluster,do=deploy): cue: marshal error: deploy.policies:
        cannot reference optional field: policies'
      name: deploy-my-component
      phase: failed
      reason: FailedAfterRetries
      type: deploy
    suspend: false
    terminated: true

The expected behavior is to run successfully without error.
Describe alternatives you've considered

Additional context

@somefive Are you working on this enhacement, if not then I would like to work on it ;)

@Somefive Are you working on this enhacement, if not then I would like to work on it ;)

This one has been solved. We forgot to close it. :)


[BUG]

Describe the bug
Customizer feature does not comply with "include" definition
Code To Reproduce Bug
/*
Include this file at the end of the Customizer variables
*/

doCrossSection = "none"; // ["none", "x", "y", "z", "-x", "-y", "-z", "polar"]
radius = 0;
phi = 0;
theta = 0;

module __Customizer_Limit__ () {}  // This actually works

_CS();

module _CS() {
    if (doCrossSection == "none") {
        d = echo("none");
        assembly();
    } else if (doCrossSection == "polar") {
        d = echo("polar");
        crossSection(r = radius, theta = theta, phi = phi)
        assembly();
    } else {
        d = echo(doCrossSection);
        crossSection(view = doCrossSection, r = radius)
        assembly();
    }
}

Expected behavior
When I include this file I would expect the customizer limit to be honored. It is not.
Include is described thusly: include  acts as if the contents of the included file were written in the including file
Screenshots
Additional context
Add any other context about the problem here.

OpenSCAD Version: 23.12.18
Other libraries used: crossSection is part of my own library

The Customizer is an OpenSCAD function not part of BOSL2.
There's another way to hide variables from the Customizer.  See https://en.wikibooks.org/wiki/OpenSCAD_User_Manual/Customizer#Creating_Tabs

My mistake. I'm so used to BOSL2 it's hard to separate it from OpenSCAD.


Error when gem install dawnscanner -P MediumSecurity

❯❯❯ gem cert --add <(curl -Ls https://raw.githubusercontent.com/thesp0nge/dawnscanner/master/certs/paolo_at_dawnscanner_dot_org.pem)
Added '/CN=paolo/DC=dawnscanner/DC=org'
❯❯❯  gem install dawnscanner -P MediumSecurity
Fetching: rainbow-2.0.0.gem (100%)
WARNING:  rainbow-2.0.0 is not signed
ERROR:  While executing gem ... (Gem::Security::Exception)
    missing signing certificate

Any ideas?

I second this issue...

Digital gem signing is a serious issue. I'm sad third party people doesn't sign their gem... I'm thinking on how to override this...

Rainbow should be removed by dependencies now.

Getting this error now
> gem install dawnscanner -P LowSecurity
WARNING:  cvss-0.99.0 is not signed
ERROR:  While executing gem ... (Gem::Security::Exception)
    missing signing certificate

Same with MediumSecurity

There's no GEM ssl signature anymore.


Use metadata IP instead of DNS name

https://github.com/rancher/rancher/issues/11275
https://github.com/rancher/rancher/issues/9894

@superseb can we make it configurable via catalog? METADATA_ADDRESS, similar to infra-templates/network-services/0/docker-compose.yml

@alena1108 As this is not a catalog item, where should I make it configurable?

This is updated to configurable address for ingress LB, tied to https://github.com/rancher/rancher-catalog/pull/1141

LGTM


Add Debian ecosystem

@rsc what do you think of this general naming convention for Linux distro packages?


CI: use cargo directly for audit target

Changes:

switches setup-audit to use cargo instead of the variable

Failed Run: https://drone-auto-casper-network.casperlabs.io/casper-network/casper-node/1309/1/5
Issue: https://app.zenhub.com/workspaces/cn-sre-60d363546afb85000e491ff6/issues/casper-network/sre/44

bors try

bors r+


Configuring operator to use gardener cert manager fails aquiring webhook-server-cert

Considering the operator chart can configure garden cert resources and the values permit overriding, I'd expect it to work on a default gardener shoot.
 <normal helm install from documentation>
 --set manager.certificates.certManager=false \
 --set manager.certificates.gardenerCertManager.issuer=garden \

Unfortunatelly the hardcoded configured cert commonName does not pass acme validation.
Warning  reconcile  4m44s (x3 over 12m)  cert-controller-manager  obtaining certificate failed: acme: error: 400 :: POST :: https://acme-v02.api.letsencrypt.org/acme/new-order :: urn:ietf:params:acme:error:rejectedIdentifier :: Error creating new order :: Cannot issue for "kubernetes": Domain name needs at least one dot (and 2 more problems. Refer to sub-problems for more information.), problem: "urn:ietf:params:acme:error:malformed" :: Error creating new order :: Domain name needs at least one dot, problem: "urn:ietf:params:acme:error:malformed" :: Error creating new order :: Domain name does not end with a valid public suffix (TLD), problem: "urn:ietf:params:acme:error:malformed" :: Error creating new order :: Domain name does not end with a valid public suffix (TLD)

Version: 0.5.1
Can you guys have a look?

After looking some more on gardener it seems that the cert-manager managed by the cluster only accepts acme / dns validations and will not work for local svc, and having a custom CA that does not get rotated is not that great.
Do you know if it is appropiate to have a second vanilla cert-manager? (even though the online docs say funkiness might occur if 2 of them are running)

Hi,
We don't support cluster configuration, if you need to configure values that are currently not present in the values.yaml feel free to request it and we'll evaluate it.
Thanks,
Keren.

It seems to work even in gardener having a vanilla cert-manager installed so not an issue.


copy translation not working

"translation copied" but it is not

I have the same issue
Pixel 2XL Android 12 DivestOS
DeepL Version 7.6

I have the same issue. I can copy just fine by highlighting, though.
Pixel 6 Pro. Android 13. DeepL 7.6.

Hi! Thanks for reporting the issue.
This issue has been fixed in v8.0.
https://github.com/sakusaku3939/DeepLAndroid/releases/tag/v8.0

This has broken again?


Update the BBG based on Open API 3.0

Description:
This BBG is currently written based on Open API 2.0. We need to rewrite it based on Open API 3.0.
The Pet Store implementation based on Open API 3.0 can be found in [1].
[1] https://github.com/OAI/OpenAPI-Specification/blob/master/examples/v3.0/petstore.yaml
Suggested Labels:

Suggested Assignees:

Affected Product Version:
OS, DB, other environment details and versions:
Steps to reproduce:
Related Issues:

Closing as we no longer maintain this repo.

Closing as we no longer maintain this repo.


add checkmarks

@ArvindHarinder1 : Thanks for your contribution! The author(s) have been notified to review your proposed change.


#82 group validation issues

Closes #82
Main change: Rule now return zero or one Violation instead of violation's list.
In java/scala I would use Rule.validate(Swagger): Optional<Violation> but in Kotlin it's idiomatic to use nullable type instead of Optional: Rule.validate(Swagger): Violation?.
I discovered many bugs/typos/inconsistencies/corner cases in existing rule's code (some of them I fixed in this PR, rest I will address in separate issues)
Sorry for gigantic PR. Every rule and rule test needed to be kotlinized to play well with nullability.

Example of output on api_tinbox.yaml:
Found the following MUST violations
===================================

Define and Assign Access Rights (Scopes)
        (paths:
                /meta/article_domains GET has no valid OAuth2 scope
                /meta/commodity_groups GET has no valid OAuth2 scope
                /queue/models GET has no valid OAuth2 scope
                /meta/colors GET has no valid OAuth2 scope
                /queue/configs/{config-id} GET has no valid OAuth2 scope
                /queue/summaries GET has no valid OAuth2 scope
                /meta/size_grids GET has no valid OAuth2 scope
                /queue/models/{model-id} GET has no valid OAuth2 scope
                /merchants GET has no valid OAuth2 scope
                /meta/tags GET has no valid OAuth2 scope
                /rejections POST has no valid OAuth2 scope
        )
        Every endpoint must be secured by proper OAuth2 scope
        http://zalando.github.io/restful-api-guidelines/security/Security.html#must-secure-endpoints-with-oauth-20

Every Second Path Level To Be Parameter
        (paths:
                /meta/article_domains
                /meta/commodity_groups
                /queue/models
                /meta/colors
                /queue/configs/{config-id}
                /queue/summaries
                /meta/size_grids
                /queue/models/{model-id}
                /meta/tags
        )
        Every second path level must be a path parameter

Lowercase words with hyphens
        (paths:
                /meta/article_domains
                /meta/commodity_groups
                /meta/size_grids
        )
        Use lowercase separate words with hyphens for path segments
        http://zalando.github.io/restful-api-guidelines/naming/Naming.html#must-use-lowercase-separate-words-with-hyphens-for-path-segments

Use snake_case (never camelCase) for Query Parameters
        (paths:
                /queue/models GET
                /queue/summaries GET
        )
        Parameters that are not in snake_case: merchant-id
        http://zalando.github.io/restful-api-guidelines/naming/Naming.html#must-use-snakecase-never-camelcase-for-query-parameters

Response As JSON Object
        (paths:
                /meta/article_domains GET 200
                /meta/commodity_groups GET 200
                /queue/models GET 200
                /meta/colors GET 200
                /queue/configs/{config-id} GET 200
                /queue/summaries GET 200
                /meta/size_grids GET 200
                /queue/models/{model-id} GET 200
                /merchants GET 200
                /meta/tags GET 200
        )
        Always Return JSON Objects As Top-Level Data Structures To Support Extensibility
        https://zalando.github.io/restful-api-guidelines/compatibility/Compatibility.html#must-always-return-json-objects-as-toplevel-data-structures-to-support-extensibility

Found the following SHOULD violations
=====================================

Limit number of Resources
        (paths:
                /meta/article_domains
                /meta/commodity_groups
                /queue/models
                /meta/colors
                /queue/configs/{config-id}
                /queue/summaries
                /meta/size_grids
                /queue/models/{model-id}
                /merchants
                /meta/tags
                /rejections
        )
        Number of paths 11 is greater than 8
        http://zalando.github.io/restful-api-guidelines/resources/Resources.html#should-limit-number-of-resources

Pluralize Resource Names
        (paths:
                /queue/models
                /queue/configs/{config-id}
                /queue/summaries
                /queue/models/{model-id}
        )
        Resources 'queue' are singular (but we are not sure)
        http://zalando.github.io/restful-api-guidelines/naming/Naming.html#must-pluralize-resource-names

Summary:
========

MUST violations: 5
SHOULD violations: 2
COULD violations: 0
HINT violations: 0

@netme I need help to figure out how to fix build (I'm not familiar with .bats tests)

@mkulak sure, I will take a look

@mkulak , I've fixed tests 2 and 4, 3 and 5 are failing due to the NullPointerException from server side:
java.lang.NullPointerException: null
        at de.zalando.zally.rules.CommonFieldNamesRule.validate(CommonFieldNamesRule.kt:19) ~[main/:na]
        at de.zalando.zally.rules.RulesValidator.validate(RulesValidator.kt:22) ~[main/:na]
        at de.zalando.zally.ApiViolationsController.validate(ApiViolationsController.java:45) ~[main/:na]
        at sun.reflect.GeneratedMethodAccessor120.invoke(Unknown Source) ~[na:na]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_77]
        at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_77]
        at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:220) ~[spring-web-4.3.5.RELEASE.jar:4.3.5.RELEASE]
        at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:134) ~[spring-web-4.3.5.RELEASE.jar:4.3.5.RELEASE]
        at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:116) ~[spring-webmvc-4.3.5.RELEASE.jar:4.3.5.RELEASE]
        at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:827) ~[spring-webmvc-4.3.5.RELEASE.jar:4.3.5.RELEASE]
        at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:738) ~[spring-webmvc-4.3.5.RELEASE.jar:4.3.5.RELEASE]
        at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85) ~[spring-webmvc-4.3.5.RELEASE.jar:4.3.5.RELEASE]
        at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:963) ~[spring-webmvc-4.3.5.RELEASE.jar:4.3.5.RELEASE]
        at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:897) ~[spring-webmvc-4.3.5.RELEASE.jar:4.3.5.RELEASE]
        at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970) [spring-webmvc-4.3.5.RELEASE.jar:4.3.5.RELEASE]
        at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:872) [spring-webmvc-4.3.5.RELEASE.jar:4.3.5.RELEASE]
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:648) [tomcat-embed-core-8.5.6.jar:8.5.6]
        at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846) [spring-webmvc-4.3.5.RELEASE.jar:4.3.5.RELEASE]
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:729) [tomcat-embed-core-8.5.6.jar:8.5.6]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:230) [tomcat-embed-core-8.5.6.jar:8.5.6]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165) [tomcat-embed-core-8.5.6.jar:8.5.6]
        at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) [tomcat-embed-websocket-8.5.6.jar:8.5.6]

To check it manually, you can run:
cli/bin/zally server/src/test/resources/fixtures/api_spp.json

@netme got it. Fixed NPE.

@mkulak cool, I will fix 2 more tests tonight / tomorrow

@netme I fixed integration tests (at least locally). It took me a while to figure out what means this
" [32m" (at first I thought it's weird bats DSL)

@mkulak thanks for fixing it. Yes, it's a bit tricky to maintain bats tests, that's why we have #108 . But it was quite fast to set it up few weeks ago. Hope we'll fix that after MS1 .

As discussed, the size of this PR is ok :)
Looks good so far (I only skimmed through, I trust you).
I would like to propose one change - regarding the output format. It looks quite technical, I'd like to change it to:
Use snake_case (never camelCase) for Query Parameters
        Parameters that are not in snake_case: merchant-id
        See http://zalando.github.io/restful-api-guidelines/naming/Naming.html#must-use-snakecase-never-camelcase-for-query-parameters
        Violated at:
                /queue/models GET
                /queue/summaries GET

What do you think?

@fmueller I totally agree.

Really nice PR, I personally like "kotlinization" part in 4270001 .

Updating branch

👍

Impressive PR!

Thanks!
I pushed output format changes. Now integration test fail.

@mkulak do you want to fix it or should I?

@mkulak @netme This is just because @mkulak fixed some rules. I adapt the integration tests. :)

Thanks!
On Wed, 1 Mar 2017 at 11:34, Felix Müller<EMAIL_ADDRESS>wrote:

@mkulak https://github.com/mkulak @netme https://github.com/netme
This is just because @mkulak https://github.com/mkulak fixed some
rules. I adapt the integration tests. :)
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/zalando-incubator/zally/pull/157#issuecomment-283304500,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAF5KK-1x_z0iNvwvCOMNE-W_xsqgYugks5rhUmbgaJpZM4MMLvO
.

--
Kind regards,
Michael Chernykh

👍

👍

Yoohoo!


fix: continue for expiration/drift when pods cannot reschedule

Fixes #N/A
Description
When pods could not all schedule, expiration and drift would continue. This asserted that nodes that were expired or drifted would eventually be cleaned up. Yet, in the case where all replacement types were exhausted, this would just result in a delete action, which if the Provisioner has no correct configuration, could result in a scale to 0 situation.
How was this change tested?
make presubmit
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

18 of 22 (81.82%)  changed or added relevant lines in 3 files are covered.
4 unchanged lines in 2 files lost coverage.
Overall coverage increased (+0.2%) to 79.637%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

pkg/utils/machine/machine.go
4
5
80.0%

pkg/controllers/deprovisioning/drift.go
11
14
78.57%

Files with Coverage Reduction
New Missed Lines
%

pkg/controllers/provisioning/scheduling/topology.go
2
86.49%

pkg/controllers/provisioning/scheduling/topologygroup.go
2
96.75%

Totals

Change from base Build<PHONE_NUMBER>:
0.2%

Covered Lines:
8076

Relevant Lines:
10141

💛  - Coveralls

@njtran  It looks like this introduced a test flake on should drift one non-empty node at a time, starting with the earliest drift.  See https://github.com/aws/karpenter-core/actions/runs/5814182481/job/15763328436


Sast

We can remove pipeline scanning from open source repos if that's easier for now. Let's revisit open source/public repo scanning after Checkmarx is launched internally


feat(vm): update anno for hotplug volumes

Problem:
When users add/remove hotplug volumes, we didn't update volumeClaimTemplates annotation, so we can't update the sizes of hotplug volumes on the VM page.
Solution:
Add/remove hotplug volumes in volumeClaimTemplates.
Related Issue:
https://github.com/harvester/harvester/issues/2810
Test plan:
Setup:

Create a harvester clsuter.
Create a VM my-vm.
Create a Volume another-volume with size 1Gi.

Case 1: Add volume button can add volume to volumeClaimTemplates annotation

Click Add Volume on the my-vm and add another-volume as disk-2.
Check whether another-volume is in harvesterhci.io/volumeClaimTemplates annotation.

Case 2: Users can resize hotplug volumes

Stop my-vm.
Change storage to 2Gi for another-volume through harvesterhci.io/volumeClaimTemplates annotation.
Check whether another-volume is changed to 2Gi.
Start my-vm.

Case 3: Remove volume button can remove volume in volumeClaimTemplates annotation

Click View in API on my-vm.
Click removeVolume and input disk-2.
Check whether another-volume is removed in harvesterhci.io/volumeClaimTemplates annotation.

@guangbochen @futuretea, I update the PR to add retry mechanism since we update VM twice in one request. The second request may have conflicts.

https://github.com/Mergifyio backport v1.1


Ensure asset manifest contents include fingerprinted paths

Fixed the true scenario - https://github.com/ember-engines/ember-engines/pull/384#issuecomment-321823354
@rwjblue @dgeb

@dgeb can we merge this?

@stefanpenner can you take a look this, please?

@stefanpenner @rwjblue  can you take a look this, please?

This looks identical to #384 🤔

no @dgeb I replace after per before following the true scenario by @rwjblue on old PR.
take a look, please - https://github.com/ember-engines/ember-engines/pull/384#issuecomment-321823354

no @dgeb I replaced after per before following the true scenario by @rwjblue on old PR.

Yeah, I finally figured it out after staring at the diff again (and deleted my prior comment). Thanks for the clarification.

Did you actually try running this with a production build for lazy engines?
This is the content of the <meta name="[app-name]/config/asset-manifest"> tag, when I use the latest master of ember-engines.
{
  "bundles": {
    "@clark/engine-cockpit-retirement": {
      "assets": [
        {
          "uri"<EMAIL_ADDRESS>          "type": "js"
        },
        {
          "uri"<EMAIL_ADDRESS>          "type": "css"
        },
        {
          "uri"<EMAIL_ADDRESS>          "type": "js"
        }
      ]
    },
    // ...
  }
}

As you can see, the URIs are not fingerprinted in there and loading the assets fails as the actual files are fingerprinted.

Interestingly the correct "fix" for me actually is #384. It only works with after, not before.

/cc @spencer516

Even more interestingly, just removing the "before": "broccoli-asset-rev" from the latest master suddenly exhibits the same issue, even though this setup has been working for months.

The reason for this was that I used a pre-published version as @buschtoens/ember-engines, which lexicographically sorts it before broccoli-asset-rev. When using something like patch-package instead, this does not occur.

I agree w/ @buschtoens, I think the correct way would be to use "after": "broccoli-asset-rev".
First, that's what ember-asset-loader says is required.
Secondly, we started having issues lazy-loading fingerprinted engine assets after upgrading our app to ember-auto-import 2.x. The filenames in the dist folder were fingerprinted, but the entries in the asset manifest (both in index.html and asset-manifest.json) were not.
After debugging our build and reading through issues in all the related addon repos, it really seemed to me that somehow the dependency bump changed the order in which the addons were run. To verify, I added "after": "broccoli-asset-rev" to ember-engine's package.json (locally w/in our node_modules folder) and now the asset manifest URIs are fingerprinted.
My guess is that the current situation (w/o specifying before or after) works by happenstance unless your dependencies change the lexicographical sorting  of the addons and the order in which they are run.
I think https://github.com/ember-engines/ember-engines/pull/384#pullrequestreview-55812394 is also correct, when using after we do not get the other features of broccoli-asset-rev, like prepending. That has been true for us even before it recently broke, but we address that by using it's generateURI() hook.
In order to use before work the way that @rwjblue describes above, I think you'd have to also add  .json to broccoli replaceExtensions, which may have unintended consequences.


Sharding into multiple IREE logical devices or queues with peered memory

This is a proposal on how to approach distribution of an MLIR program across multiple homogeneous physical devices that have peered memory. It is intended to improve memory usage under NUMA. Devices/Queues have direct peer-to-peer memory accesses during kernel execution. We want to explicitly shard tensor data and operations and assign them to devices/queues.
Tiling and distribution of work on multiple compute units treated as a single device and queue differs for this problem in that there each work item does not matter to what compute unit it is assigned. The compiler does not have to explicitly resolve the assignment. It is left up to the runtime. Also memory allocations does not have to distinguish where they are performed.
Here we use a program with Mesh dialect sharding annotations as an input. The original intent of the Mesh dialect is to utilize MPI-like collectives instead of placement operations like extracting and inserting tensor slices. We would like to expand its scope to support the latter case also. The algorithm that produces the sharding annotation is out-of-scope here and it is assumed that the program is fully annotated.
Below is an example of an annotated program that does matrix multiplication. It is sharded on a 1D process mesh of size 2.
    2x3              3x2             2x2
+-----------+   +------------+   +-----------+
| process 0 |   | fully      |   | process 0 |
+-----------+ X | replicated | = +-----------+
| process 1 |   |            |   | process 1 |
+-----------+   +------------+   +-----------+

mesh.mesh @mesh(shape = 2, peered_memory_axes = [0])

func.func @main(
  %arg0 : tensor<2x3xf32>,
  %arg1 : tensor<3x2xf32>,
  %out_dps: tensor<2x2xf32>
) -> tensor<2x2xf32> {
  %arg0_sharded = mesh.shard %arg0 to <@mesh, [[0]]> : tensor<2x3xf32>
  %arg1_sharded = mesh.shard %arg1 to <@mesh, []> : tensor<3x2xf32>
  %out_dps_sharded = mesh.shard %arg1 to <@mesh, [0]> : tensor<2x2xf32>

  %res = linalg.matmul ins(%arg0_sharded, %arg1_sharded :
    tensor<2x3xf32>, tensor<3x2xf32>)
    outs(%out_dps_sharded : tensor<2x2xf32>) -> tensor<2x2xf32>

  %res_sharded = mesh.shard %res to <@mesh, [[0]]> : tensor<2x2xf32>

  return %res_sharded : tensor<2x2xf32>
}

Note that the mesh shape may be dynamic.
For example
mesh.mesh @mesh(shape = ?x?x?, peered_memory_axes = [1, 2])

Here we have a 3D mesh with all dimensions being dynamic. It has peered memory across mesh axes 1 and 2. The mesh is partitioned into disjoint subsets where there is uniform memory only within a subset. Processes (h, i, j) and (k, l, m) have peered memory for h = k and distinct memory for h != k. In the immediate future we will be concerned with 1D meshes with peered memory.

As a first step we transform the program from annotations using mesh.shard operations to a distributed tensor form that utilizes the tensor encoding attribute.
// Shard tensor axis 0 on mesh axis 0.
#shard_0_on_0 = #mesh.shard<@mesh, [[0]]>
// The tensor is replicated on all shards.
#fully_replicate = #mesh.shard<@mesh, []>

mesh.mesh @mesh(shape = 2, peered_memory_axes = [0])

func.func @main(
  %arg0 : tensor<2x3xf32, #shard_0_on_0>,
  %arg1 : tensor<3x2xf32, #fully_replicate>,
  %out_dps: tensor<2x2xf32, #shard_0_on_0>
) -> tensor<2x2xf32, #shard_0_on_0> {
  %res = linalg.matmul ins(%arg0, %arg1 :
    tensor<2x3xf32, #shard_0_on_0>,
    tensor<3x2xf32, #fully_replicate>)
    outs(%out_dps : tensor<2x2xf32, #shard_0_on_0>)
    -> tensor<2x2xf32, #shard_0_on_0>

  return %res : tensor<2x2xf32, #shard_0_on_0>
}

The usage of the encoding attribute may clash with other uses like sparsity encoding. We access the mesh sharding attribute through a type interface that hides the encoding attribute detail to make future changes easier.
In this step we would insert resharding if required by the annotations. In general under peered memory when access uniformity is high we would like to avoid resharding and have the subsequent operation be able to consume the original sharding.

After converting to distributed tensors we shard the operations, where we describe what each process is doing. One approach is to utilize a mesh.for_all operation. All processes share the same code. It is similar to the flow.dispatch.workgroups.
#shard_0_on_0 = #mesh.shard<@mesh, [[0]]>
#fully_replicate = #mesh.shard<@mesh, [[]]>

mesh.mesh @mesh(shape = 2, peered_memory_axes = [0])

func.func @main(
  %arg0 : tensor<2x3xf32, #shard_0_on_0>,
  %arg1 : tensor<3x2xf32, #fully_replicate>,
  %out_dps: tensor<2x2xf32, #shard_0_on_0>
) -> tensor<2x2xf32, #shard_0_on_0> {

  %res = mesh.for_all on @mesh ->
    tensor<2x3xf32, #shard_0_on_0> {
    %c2 = arith.constant 2 : index
    %proc_idx = mesh.process_multi_index on @mesh : index
    %slice_offset = arith.muli %proc_idx, %c2 : index
    %arg0_slice = tensor.extract_slice %arg0[%slice_offset, 0][1, 3][1, 1] :
      tensor<2x3xf32, #shard_0_on_0> to tensor<1x3xf32>
    %out_dps_slice = tensor.extract_slice %out_dps[%slice_offset, 0][1, 2][1, 1] :
      tensor<2x2xf32, #shard_0_on_0> to tensor<1x2xf32>

    %res_slice = linalg.matmul ins(%arg0_slice, %arg1 :
      tensor<1x3xf32>,
      tensor<3x2xf32, #fully_replicate>)
      outs(%out_dps_slice : tensor<2x2xf32, #shard_0_on_0>)
      -> tensor<1x2xf32>
    // We return the shard for this process.
    return %res_slice : tensor<1x2xf32>
  }

  return %res : tensor<2x2xf32, #shard_0_on_0>
}

Instead of a new mesh.for_all operation we can introduce mesh.for_one. It describes what each process is doing separately. With this approach we combine it with scf.forall to describe the operations for the entire mesh.
#shard_0_on_0 = #mesh.shard<@mesh, [[0]]>
#fully_replicate = #mesh.shard<@mesh, [[]]>

mesh.mesh @mesh(shape = 2, peered_memory_axes = [0])

func.func @main(
  %arg0 : tensor<2x3xf32, #shard_0_on_0>,
  %arg1 : tensor<3x2xf32, #fully_replicate>,
  %out_dps: tensor<2x2xf32, #shard_0_on_0>
) -> tensor<2x2xf32, #shard_0_on_0> {
  %c0 = arith.constant 0 : index
  %c1 = arith.constant 1 : index
  %mesh_size = mesh.mesh_shape @mesh : index
  %res = scf.forall (%proc_idx) in (%mesh_size)
    shared_outs(%out = %out_dps) -> (tensor<2x2xf32, #shard_0_on_0>) {
      scf.forall.in_parallel {
      %res = mesh.for_one %proc_idx on @mesh {
        %c2 = arith.constant 2 : index
        %slice_offset = arith.muli %proc_idx, %c2 : index
        %arg0_slice = tensor.extract_slice %arg0[%slice_offset, 0][1, 3][1, 1] :
          tensor<2x3xf32, #shard_0_on_0> to tensor<1x3xf32>
        %out_slice = tensor.extract_slice %out[%slice_offset, 0][1, 2][1, 1] :
          tensor<2x2xf32, #shard_0_on_0> to tensor<1x2xf32>

        %res_slice = linalg.matmul ins(%arg0_slice, %arg1 :
          tensor<1x3xf32>,
          tensor<3x2xf32, #fully_replicate>)
          outs(%out_slice : tensor<2x2xf32, #shard_0_on_0>)
          -> tensor<1x2xf32>

        tensor.parallel_insert_slice %res_slice into
          %out[%slice_offset, 0][1, 2][1, 1] :
          tensor<1x2xf32> into tensor<2x2xf32, #shard_0_on_0>
      }
    }
  }

  return %res : tensor<2x2xf32, #shard_0_on_0>
}

After all operations have been sharded we need convert each distributed tensor into a list of tensors.
In the case of a dynamic mesh we need to use a dynamic list. We can utilize tensor<?x...>. This dynamic nature will require propagation all the way to the runtime. The VM would need to handle dynamic lists of buffers.
In the case of a static mesh we can use the values that correspond to shards directly. In this case we have to also unroll all loops that run operations on the mesh. Bellow is an example of doing that.
mesh.mesh @mesh(shape = 2, peered_memory_axes = [0])

func.func @main(
  %arg0_shard0 : tensor<1x3xf32>,
  %arg0_shard1 : tensor<1x3xf32>,
  %arg1_shard0 : tensor<3x2xf32>,
  %arg1_shard1 : tensor<3x2xf32>,
  %out_dps_shard0: tensor<1x2xf32>,
  %out_dps_shard1: tensor<1x2xf32>,
) -> (tensor<1x2xf32>, tensor<1x2xf32>) {
  // Shard 0.
  %res_shard0 = mesh.for_one 0 on @mesh {
    %res = linalg.matmul ins(%arg0_shard0, %arg1_shard0 :
      tensor<1x3xf32>, tensor<3x2xf32>)
      outs(%out_dps_shard0 : tensor<1x2xf32>)
      -> tensor<1x2xf32>
    return %res : tensor<1x2xf32>
  }

  // Shard 1.
  %res_shard1 = mesh.for_one 1 on @mesh {
    %res = linalg.matmul ins(%arg0_shard1, %arg1_shard1 :
      tensor<1x3xf32>, tensor<3x2xf32>)
      outs(%out_dps_shard2 : tensor<1x2xf32>)
      -> tensor<1x2xf32>
    return %res : tensor<1x2xf32>
  }

  return %res_shard0, %res_shard0 : tensor<1x2xf32>, tensor<1x2xf32>
}

This example is manicured for illustration. In real-world programs a mesh process may have to ingest operands that require extracting slices that fall on the boundary of tensor shards. If we can reliably fuse mesh.for_one operations to avoid copying, operand resharding can be handled as a separate concern.
At this point we are ready to convert the remaining mesh.for_one ops into IREE specific descriptions of device/queue affinity.

We could shard after dispatch region formation, but before transforming into workgroups. It is expected that the user may have provided some sharding annotations throughout the program. Then we form the dispatch regions, complete the sharding annotations and lastly partition into Mesh processes. The user-provided sharding annotations may hinder fusion of the dispatch regions. Can we place sharding annotations inside dispatch regions and what does that even mean? This approach also requires that sharding propagation handles dispatch regions. This drives the requirement for solving the more general problem of propagating through a composite operation.
The other option is to shard before forming the dispatch regions. In this scenario during dispatch region formation we will have to handle a lot of tensor placement (insert/extract) operations due to resharding. It would also have to handle device/queue placement.
We could expand the scope of flow.dispatch.workgroups to handle additional dimensions that are mapped to devices/queues.

We need to provide the user functionality to shard/unshard function arguments/results. This should be coupled with function variants that reserve the original signature and handle all sharding inside.
func.func @main_shard_arg0(%arg0 : tensor<2x3xf32>)
  -> tensor<2x3xf32, #shard_0_on_0> {
  %c0 = arith.constant 0 : index
  %c1 = arith.constant 1 : index
  %mesh_size = mesh.mesh_shape @mesh : index
  %out_dps = tensor.empty : tensor<2x3xf32, #shard_0_on_0>
  %res = scf.forall (%proc_idx) in (%mesh_size)
    shared_outs(%out = %out_dps) -> (tensor<2x3xf32, #shard_0_on_0>) {
      scf.forall.in_parallel {
      %res = mesh.for_one %proc_idx on @mesh {
        %c2 = arith.constant 2 : index
        %slice_offset = arith.muli %proc_idx, %c2 : index
        %slice = tensor.extract_slice %arg0[%slice_offset, 0][1, 3][1, 1] :
          tensor<2x3xf32> to tensor<1x3xf32>

        tensor.parallel_insert_slice %slice into
          %out[%slice_offset, 0][1, 3][1, 1] :
          tensor<1x3xf32> into tensor<2x3xf32, #shard_0_on_0>
      }
    }
  }

  return %res : tensor<2x3xf32, #shard_0_on_0>
}

func.func @main(%arg0 : tensor<2x3xf32>, ...) ... {
  %arg0_sharded = func.call @main_shard_arg0(%arg0) : (tensor<2x3xf32>) -> tensor<2x3xf32, #shard_0_on_0>
  ...
  %res = func.call @main_shrded(%arg0_sharded, ...) ...
  // unshard %res and return
}

The resulting IR can be handled as the rest of the program to convert each distributed tensor into a list of tensors.
How to stream the tensor shards from the host directly into the destination slices without having to allocate the original unsharded tensor?

There is a potential optimization for some operations that have high computational complexity like matrix multiplication. When tiled, in order to avoid read multiple times the same tile from a slow memory region in the kernel, each such tile can be copied closer to the compute unit. We don't want to do that before starting computation, but to interleave it into the kernel.

I would like for some feedback on what and if this makes sense.
@benvanik, @stellaraccident could you take a look?

Great examples - really helps ground things! I will have to take a longer read as there's lots to unpack - but I think there's a layering issue here I need to put my finger on.
The immediate feedback is that we don't want to use tensor encodings for placements - we need to keep those for data layout/encodings - putting mesh dialect-specific encoding behavior between frontends and the rest of the compiler makes it practically impossible to carry the information we need. We also need to be able to mutate layouts at several points in the lowering pipeline and cannot practically preserve mixed semantics outside of very specific slices of the pipeline (which then ends up with a lot of trickery to convert into/out-of without losing information/breaking). So any solution we have will need to use ops instead of attributes to indicate sharding/transfers.
Dynamic VM lists of distributed tensors and such also raises some flags for me as that's going to not be possible in flow/stream as they exist today. Nor will usage of scf.forall and other functionality be trivial (though it'd be cool for other purposes). This could be a year+ of work and a lot of refactoring/rewriting and my feeling is that we're going to want to find solutions that let us progress much earlier/more incrementally. That may mean not using the mesh dialect - I'm not quite sure what it's adding here relative to the cost to support it.

Thanks for the very explicit examples. I'm still digesting.
I think what I'm struggling with is at what level we break the spend abstraction. With this approach, it is very late, hanging on to the dtensor paradigm through most of the pipeline.
Whereas in frameworks like tf dtensor and pytorch fsdp, it happens very early, at least for the parts running at the lowest level of the hierarchy on the same bus. Doing it late like this works for the homogenous distributed case. But I wonder how it would interplay to just start with some higher level operators that indicated device placement and movement (since that is already needed for heterogenous).
I don't think I'm saying this perfectly -- but I think there may be two concerns here, not just one and I'm trying to put my finger in it.

But I wonder how it would interplay to just start with some higher level operators that indicated device placement and movement (since that is already needed for heterogenous).
@stellaraccident

For a heterogenous system the device mesh representation would not be sufficient. I that case you would probably want a hierarchical representation like a file system.
E.g.
    -------- node_0 ----------  ...  node_k
   /        /      \          \
cpu_0 ... cpu_m    gpu_0 ... gpu_n
                  /    \
         chiplet_0 ... chiplet_q

Then we can assign properties on each level, like peered memory or communication bandwidth.

Here's an older issue tracking the need for multiple physical devices per logical device exposed as queues: #10765


Cannot call method 'toString' of undefined/User info not passed to adapter.

Hello,
We have been trying to set up slack on hubot. We are following this gist
This is what is done so far:

Create hubot user on slack
launch hubot instance

HUBOT_IRC_NICK="ibot" \
HUBOT_IRC_SERVER="company.irc.slack.com" \
HUBOT_IRC_DEBUG=true \
HUBOT_IRC_PASSWORD="company.randompassword" \
HUBOT_IRC_ROOMS="#test,#general,#team-support" \
bin/hubot -a irc --name ibot

This launches the bot and we can see the user online. But, the bot throws out an error every time there is a message:
[Wed Nov 12 2014 10:45:54 GMT+0530 (IST)] ERROR TypeError: Cannot call method 'toString' of undefined
  at new User (/home/code/ibotv3/slack-ibot/node_modules/hubot/src/user.coffee:9:19, <js>:14:47)
  at Brain.userForId (/home/code/ibotv3/slack-ibot/node_modules/hubot/src/brain.coffee:106:7, <js>:93:16)
  at IrcBot.getUserFromId (/home/code/ibotv3/slack-ibot/node_modules/hubot-irc/src/irc.coffee:106:12, <js>:175:33)
  at IrcBot.createUser (/home/code/ibotv3/slack-ibot/node_modules/hubot-irc/src/irc.coffee:112:12, <js>:182:19)
  at Client.<anonymous> (/home/code/ibotv3/slack-ibot/node_modules/hubot-irc/src/irc.coffee:249:14, <js>:333:21)
  at Client.EventEmitter.emit (events.js:106:17)
  at Client.<anonymous> (/home/code/ibotv3/slack-ibot/node_modules/hubot-irc/node_modules/irc/lib/irc.js:471:22)
  at Client.EventEmitter.emit (events.js:95:17)
  at /home/code/ibotv3/slack-ibot/node_modules/hubot-irc/node_modules/irc/lib/irc.js:633:22
  at Array.forEach (native)
  at Socket.<anonymous> (/home/code/ibotv3/slack-ibot/node_modules/hubot-irc/node_modules/irc/lib/irc.js:630:15)
  at Socket.EventEmitter.emit (events.js:95:17)
  at Socket.<anonymous> (_stream_readable.js:746:14)
  at Socket.EventEmitter.emit (events.js:92:17)
  at emitReadable_ (_stream_readable.js:408:10)
  at emitReadable (_stream_readable.js:404:5)
  at readableAddChunk (_stream_readable.js:165:9)
  at Socket.Readable.push (_stream_readable.js:127:10)
  at TCP.onread (net.js:526:21)

version info:
"dependencies": {
    "hubot": "^2.9.3",
    "hubot-diagnostics": "0.0.1",
    "hubot-google-images": "^0.1.0",
    "hubot-google-translate": "^0.1.0",
    "hubot-help": "^0.1.1",
    "hubot-heroku-keepalive": "0.0.4",
    "hubot-irc": "^0.2.7",
    "hubot-maps": "0.0.0",
    "hubot-pugme": "^0.1.0",
    "hubot-rules": "^0.1.0",
    "hubot-scripts": "^2.5.16",
    "hubot-shipit": "^0.1.1",
    "hubot-youtube": "^0.1.2"
  },
  "engines": {
    "node": "0.10.x"
  }

I can see that the from parameter is not passed on to the message listener
relevant line
Not sure if this is a problem on the slack's side or there is a bug in the adapter. Any help with debugging the issue would be appreciated.

I have the exact same error.

Just started getting this today after Hubot was working fine (through Slack) for the past few months. Nothing changed on Hu's end.

I'm also getting this error, and after setting HUBOT_LOG_LEVEL to debug it seems that the error is coming from the robot seeing its own message, or if a message comes from a user it can't parse.
This is the output from hubot using hubot-jira-lookup (triggering off my message IT-3402), then spitting out a message to the channel. When the script outputs through the robot the user is sent through as undefined:
[Thu Apr 30 2015 10:31:35 GMT-0700 (PDT)] INFO #discovery <kevin> yes
[Thu Apr 30 2015 10:31:35 GMT-0700 (PDT)] INFO #discovery <kevin> IT-3402
[Thu Apr 30 2015 10:31:35 GMT-0700 (PDT)] DEBUG Message 'IT-3402' matched regex //\b[A-Z]{2,5}-[0-9]{1,5}\b//
[Thu Apr 30 2015 10:31:37 GMT-0700 (PDT)] DEBUG From undefined to #discovery: Web servers full of mysterious event logs
[Thu Apr 30 2015 10:31:37 GMT-0700 (PDT)] ERROR TypeError: Cannot call method 'toString' of undefined


fix(deps): switch to scancode mini

Removes amd64-only pre-built binary dependencies.
scancode-toolkit ships with typecode-libmagic which is not built for arm64, thus causing a build error.
This PR switches to the scancode-toolkit-mini which does not package the binaries. The system dependency libmagic is installed separately instead.

Looks great and awesome to see the action triggering correctly 🤩


feat : Verify No New Console Errors

Overall
Introduce a diffing automation tool in Circle CI that uses logged console errors to compare past , present, and future console errors.
Summary
By looking at previous jobs and comparing outputs with errors one can triage common blunders or errors.

Per : https://circleci.com/docs/enterprise/logging/
Discussed with @griggheo to trial out the approach of:
CircleCI -> DynamoDB rather than CircleCI -> CloudWatch -> SQS
Due to SQS short life cycle.


refill_address_queue key error

Hi, I am receiving the following error when i start my celery worker
KeyError: 'wallet.tasks.refill_addresses_queue'
I have my broker running, as well as bitcoind, and have an instance of my currency model within my database.
Do i require the django-celery backend in order to save the results of the task to the DB?
thanks

Apologies I read through other resolved issues and it solver my problem


Support for Ethereum 2.x

There is something that prevents support for ethereum==2.x ?
Thanks in advance :rocket:

I'm trying to use python3.6 with ethereum-2.x.x (i need ethereum.tools) and I am using Populus ("testrpc" chain) for testing. This causes a dependency on eth_testrpc library which has the dependency on ethereum-1.6.1 (pyethereum).

Yes @johannes-schuetze i was facing something similar when using eth_testrpc.
If you use Docker, take a look at docker pull lefortpablo/ethereum-test


Training detail

Hi, really impressive work! I'm curious why using zero3 in pretraining but zero2 in instruction training?

We find using zero3 is very easy to hang because the lengths of the video and image data vary greatly.

Yes, it's true. Besides, in the instruction training phase, I met  CUDA out of memory error in 5258/8056. Are your experiments conducted on A100 GPUs with 40GB or 80GB


Return a keyPathInParent for syntax collections

Previously, keyPathInParent returned nil for nodes that occurred in SyntaxCollections. Return a subscript-based key path instead.
rdar://111944659

@swift-ci Please test

@swift-ci Please test

@swift-ci Please test Windows

@swift-ci Please test

@swift-ci please test Windows

Windows failure does actually look related unfortunately. Though more like a compiler bug than an issue with this PR 😅

This is triggering SIL verification failures on Windows (https://github.com/apple/swift/67676). Marking as draft until that’s fixed

@swift-ci Please test Windows


Fix tests after re-arrangement of redirects for tagging spreadsheets

PR #123  changed the redirects of some controller actions in order to improve the user experience.
Unfortunately, Jenkins didn't run the tests for some reason and I ended up merging the PR too soon.
Some of the tests were broken because they relied on a flow that doesn't exist anymore, so this commit fixes those tests.

@mobaig I've pushed the change including the explicit paths.

Noice.


NFDIV-758: Update content on joint submission page

JIRA link
https://tools.hmcts.net/jira/browse/NFDIV-758

Closed as ticket is for a new page - not the existing joint submission page


[translation] Add Spanish translation

Fixes #424
Spanish translation
Context

Messages translated to Spanish (es)

Checklist

[ ] Review Contribution Guidelines.
[ ] Make sure all contributed code can be distributed under the terms of the
Apache License 2.0, e.g. the code was written by
you or the original code is licensed under a license compatible to Apache License 2.0.
[ ] Update documentation when applicable.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

@torrespro Thank you! Would you mind reading and signing the CLA?

Thank you! 😄


[Question] About the results on ScienceQA in LLaVA1.5

Question
Hello,sorry to disturbe
I tried to conduct the scienceQA test on llava1.5 and found that the accuracy rate on minitest.jasol was 71.58%, which was consistent with the accuracy in the paper, but the result on test.jasonl was 70.25%. I don’t know if the 71.6% in the paper refers to minitest. result, thank you!

Question
Hello,sorry to disturbe I tried to conduct the scienceQA test on llava1.5 and found that the accuracy rate on minitest.jasol was 71.58%, which was consistent with the accuracy in the paper, but the result on test.jasonl was 70.25%. I don’t know if the 71.6% in the paper refers to minitest. result, thank you!

The results I reproduced: IMG-Accurarcy of 70.85% on the test set, with A100 GPUs.

你好，邮件我已收到，我会尽快回复哒！

The results for llava-v1.5-7b I reproduced:

total 4241, Correct: 2780, Accuracy: 65.55%, Img-Accuracy: 63.91%
question-file: llava_test_CQM-A.json

63.91% is less than 66.8% in paper for llava-v1.5-7b

I just tried and I am able to reproduce the ScienceQA results from scratch:

Clone the repo on GitHub
Create a completely new conda environment
pip install -e .
Download eval.zip from Google Drive and unzip
Clone ScienceQA repo, and download images
run bash scripts/v1_5/eval/sqa.sh

First, the split is test and not minitest, as you can see from the annotation file length.

Second, comment out the first part of the sqa.sh and use the provided results, do you reproduce the number below?

bash scripts/v1_5/eval/sqa.sh
Total: 4241, Correct: 3155, Accuracy: 74.39%, IMG-Accuracy: 71.64%

Third, uncomment, download the answers/results, and run sqa.sh

Note that when I uploaded eval.zip results, I generated the predictions using A100, and tonight I evaluated using A6000, and the numbers are exactly the same, so I am pretty confident that the numbers are correct.
@jiangsongtao @jameszhou-gl @yzbx
Can you please check and let me know in what steps that may be different from the steps above?

for llava-v1.5-13b, I reproduce the result of @jameszhou-gl
total = 4241, Correct: 2982, Accuracy: 70.31%, Img-Accuracy: 70.85%, lower than 71.6% in paper.
note I use temperature=0 in llava/eval/model_vqa_science.py
--model-path: liuhaotian/llava-v1.5-13b
--question-file: llava_test_CQM-A.json,
with A800 GPUs.

@yzbx Can you provide the full script you used?

@haotian-liu I use the follow scripts.
for split in test
do
    export TRANSFORMERS_OFFLINE=1
    export PATH=/opt/conda/bin:$PATH
    export PYTHONPATH=.
    cd /mydata/git/LLaVA-main
    prompt="CQM-A"
    model=13b
    torchrun --nnodes 1 --nproc-per-node 8 --master-port 12356 llava/eval/model_vqa_science.py \
    --model-path /mydata/open_vlm/llava-v1.5-${model}/ \
    --question-file /mydata/dataset/ScienceQA/llava_${split}_${prompt}.json \
    --image-folder /mydata/dataset/ScienceQA/${split} \
    --answers-file results/ScienceQA/${split}_${prompt}_llava-${model}.jsonl \
    --single-pred-prompt \
    --temperature 0 \
    --conv-mode vicuna_v1 \
    --tmp-dir tmp/${split}

    python llava/eval/eval_science_qa.py \
        --base-dir /mydata/dataset/ScienceQA/ \
        --result-file results/ScienceQA/${split}_${prompt}_llava-${model}.jsonl \
        --output-file results/ScienceQA/${split}_${prompt}_llava-${model}_output.json \
        --output-result results/ScienceQA/${split}_${prompt}_llava-${model}_result.json \
        --split ${split}
done

Note: the source data llava_test_CQM-A.json is from docs/ScienceQA.md, not from eval.zip
python scripts/convert_sqa_to_llava.py \
    convert_to_llava \
    --base-dir /path/to/ScienceQA/data/scienceqa \
    --prompt-format "CQM-A" \
    --split test

Can you try using the file from the eval.zip?

@haotian-liu I found the problem
b54d1eca943503d24bbed807b78f3d15  scienceqa/llava_test_CQM-A.json  # from eval.zip
9776345f346d6082a5a31c0b291e6e21  data/scienceqa/llava_test_CQM-A.json  # from git
9776345f346d6082a5a31c0b291e6e21  /data/BD/dataset/ScienceQA/llava_test_CQM-A.json  # generate with docs/ScienceQA.md

use the source data from eval.zip
I reproduce the result in paper
Total: 4241, Correct: 3155, Accuracy: 74.39%, IMG-Accuracy: 71.64%

Thanks for all your reply,i will close this issue


Disaster recovery fails with etcd v3

Rancher versions:
rancher/server:v1.6.8-rc1
rancher/agent:
Infrastructure Stack versions:
kubernetes (if applicable):1.7.2-rancher8-etcdv3
rancher/etcd:v3.0.17
**Docker version: (docker version,docker info preferred)**1.12.6
**Operating system and kernel: (cat /etc/os-release, uname -r preferred)**Ubuntu
**Type/provider of hosts: (VirtualBox/Bare-metal/AWS/GCE/DO)**DO
**Setup details: (single node rancher vs. HA rancher, internal DB vs. external DB)**Single node
**Environment Template: (Cattle/Kubernetes/Swarm/Mesos)**Kubernetes
Steps to Reproduce:

Create a 3 node Kubernetes setup and create some pods/rcs/ingress
Power OFF two hosts which results in a disaster recovery scenario
Perform Disaster recovery.

Results
Disaster recovery fails and there is no healthy etcd cluster formed. The running etcd container on which the DR was performed also errored out.

bash-4.3# etcdctl cluster-health
cluster may be unhealthy: failed to list members
Error:  client: etcd cluster is unavailable or misconfigured
error #0: dial tcp <IP_ADDRESS>:2379: getsockopt: connection refused
error #1: dial tcp <IP_ADDRESS>:4001: getsockopt: connection refused

etcd logs
8/10/2017 1:39:58 PM2017-08-10 20:39:58.045318 I | membership: added member c161b229d9ec24e8 [http://<IP_ADDRESS>:2380] to cluster 5dcde0088b02 from store
8/10/2017 1:39:58 PM2017-08-10 20:39:58.045352 I | membership: added member dc4085a3bd6ef2c8 [http://<IP_ADDRESS>:2380] to cluster 5dcde0088b02 from store
8/10/2017 1:39:58 PM2017-08-10 20:39:58.045383 I | membership: added member 288fb1180572ae34 [http://<IP_ADDRESS>:2380] to cluster 5dcde0088b02 from store
8/10/2017 1:39:58 PM2017-08-10 20:39:58.045420 I | membership: set the cluster version to 3.0 from store
8/10/2017 1:39:58 PM2017-08-10 20:39:58.049376 I | etcdmain: stopping listening for client requests on localhost:2379
8/10/2017 1:39:58 PM2017-08-10 20:39:58.049539 I | etcdmain: stopping listening for peers on http://localhost:2380
8/10/2017 1:39:58 PM2017-08-10 20:39:58.049621 C | etcdmain: database file (/pdata/data.20170810.203957.DR/member/snap/db) of the backend is missing
8/10/2017 1:39:58 PMGet http://<IP_ADDRESS>:2379/health: dial tcp <IP_ADDRESS>:2379: getsockopt: connection refused
8/10/2017 1:40:00 PMGet http://<IP_ADDRESS>:2379/health: dial tcp <IP_ADDRESS>:2379: getsockopt: connection refused

Tested on master with Kubernetes 1.7.4-rancher2-etcdv3 on a 3 node setup. Disaster recovery succeeds. etcd cluster is healthy and creation of new pods and ingress is successful in the setup.
bash-4.3# etcdctl cluster-health
member 2ab0d867a6aadf0b is healthy: got healthy result from http://<IP_ADDRESS>
:2379
member e7e0ca9ba4827bc6 is healthy: got healthy result from http://<IP_ADDRESS>
:2379
member fa6685fa31863613 is healthy: got healthy result from http://<IP_ADDRESS>:
2379
cluster is healthy
bash-4.3#

I'm seeing a similar thing happening in v1.6.14. Just this error repeatedly in the new etcd nodes after running disaster recovery.
08/03/2018 17:32:23Get http://<IP_ADDRESS>:2379/health: dial tcp <IP_ADDRESS>:2379: getsockopt: no route to host```


Decide on URLs: Content Perspective

We should sort out naming conventions early on. Needs discussion

Meeting held. Draft standards created based on today's discussion and out to team for feedback before sharing with the cannabis team.

Majority of team has provided input and approved the standards. Additional tickets can be generated if needed. Closing this ticket to reflect today's work.


double quotes " can break decoding

the following simplified original version CANNOT be parsed correctly because of the closing quote in the "From: " line.
Delivered-To<EMAIL_ADDRESS>Date: Thu, 10 Sep 2020 09:29:57 -0400
To<EMAIL_ADDRESS>From: "Amway =?utf-8?q?=C3=96sterreich"?=<EMAIL_ADDRESS>Subject: Amway Newsletter Nr. 18 - 10. September 2020
Message-ID<EMAIL_ADDRESS>Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: Quoted-Printable

if I move the closing quote after ?=, it works.
Delivered-To<EMAIL_ADDRESS>Date: Thu, 10 Sep 2020 09:29:57 -0400
To<EMAIL_ADDRESS>From: "Amway =?utf-8?q?=C3=96sterreich?="<EMAIL_ADDRESS>Subject: Amway Newsletter Nr. 18 - 10. September 2020
Message-ID<EMAIL_ADDRESS>Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: Quoted-Printable

please fix that so the parser can handle this.

Hi @markusramsak --
A quoted part takes precedence.  Specifically, "An 'encoded-word' MUST NOT appear within a 'quoted-string'.", see https://tools.ietf.org/html/rfc2047#section-5
I believe what you're trying to say is the mime-encoded part isn't "decoded", but that's correct behaviour as far as I'm aware.  It would be hard to build an exception for what you want without breaking what should be considered valid because the quotes are supposed to take precedence at least as far as I can tell.
Feel free to correct me with relevant examples, including handling by popular mail parsers or clients, or rfcs or other libraries that specifically are handling your situation differently to facilitate a discussion about it.

Hi @markusramsak --
A quoted part takes precedence.  Specifically, "An 'encoded-word' MUST NOT appear within a 'quoted-string'.", see https://tools.ietf.org/html/rfc2047#section-5
I believe what you're trying to say is the mime-encoded part isn't "decoded", but that's correct behaviour as far as I'm aware.  It would be hard to build an exception for what you want without breaking what should be considered valid because the quotes are supposed to take precedence at least as far as I can tell.
Feel free to correct me with relevant examples, including handling by popular mail parsers or clients, or rfcs or other libraries that specifically are handling your situation differently to facilitate a discussion about it.

I know that it shouldn't happen but I am the programmer of a mail client with more than 100.000 mails to parse and display and the only thing I can say is, it happens.
I just simplified the mail but the issue is real in every newsletter email from the company Amway (https://www.amway.at)
Other mail clients like gmail oder Apple Mail could decode this mail subject correctly - and I would like too.
Maybe it is just a matter of replacing "?=[space] by ?="[space] but I don't know if it would break anything

I know that it shouldn't happen but I am the programmer of a mail client with more than 100.000 mails to parse and display and the only thing I can say is, it happens.
I just simplified the mail but the issue is real in every newsletter email from the company Amway (https://www.amway.at)
Other mail clients like gmail oder Apple Mail could decode this mail subject correctly - and I would like too.
Maybe it is just a matter of replacing "?=[space] by ?="[space] but I don't know if it would break anything

Unfortunately the way the parser works, the 'part looking for quotes' is separate from the 'part looking for mime encoded parts'.  It's semantically okay for a mime-encoded part to have a quote in it, it just won't be handled as a 'control character' terminating (or starting) a quoted-part.

Unfortunately the way the parser works, the 'part looking for quotes' is separate from the 'part looking for mime encoded parts'.  It's semantically okay for a mime-encoded part to have a quote in it, it just won't be handled as a 'control character' terminating (or starting) a quoted-part.

if it can't be done on your side, than I would implement on my side to replace these wrong characters in the "From " line before it is parsed by your parser.
I would call it "preparsing" because it happens before your complex parsing.

if it can't be done on your side, than I would implement on my side to replace these wrong characters in the "From " line before it is parsed by your parser.
I would call it "preparsing" because it happens before your complex parsing.

by the way you did an excellent job with this library! About 9995 out of 10000 emails can be parsed on average from my web mail client (backed by your library) without any issues.

by the way you did an excellent job with this library! About 9995 out of 10000 emails can be parsed on average from my web mail client (backed by your library) without any issues.

if it can't be done on your side, than I would implement on my side to replace these wrong characters in the "From " line before it is parsed by your parser.

I'm not sure that it can't, but it would be an effort -- I'd have to change the precedence of how things are parsed, which would make some valid but extremely unlikely cases invalid, like From: "My =?utf-8?Q?"weird"?= name"<EMAIL_ADDRESS>(i.e. purposely containing what looks like a mime-encoded part in a name) but I can't imagine that would ever be an issue... there may be other things affected too because of how the parsers are built, it would have to be investigated.
If you're able to sanitize for exceptions you know of like that, I think that would be the way to go at least for now... we can leave this open and look when there's time or if it's affecting more people.  You could also try emailing the folks at Amway to tell them there's an issue with their emails :)  maybe they're using a house-built system that needs to be patched, or maybe it's a huge commercial system that means handling this scenario should be prioritized.

by the way you did an excellent job with this library! About 9995 out of 10000 emails can be parsed on average from my web mail client (backed by your library) without any issues.

Excellent, very happy to hear that!

if it can't be done on your side, than I would implement on my side to replace these wrong characters in the "From " line before it is parsed by your parser.

I'm not sure that it can't, but it would be an effort -- I'd have to change the precedence of how things are parsed, which would make some valid but extremely unlikely cases invalid, like From: "My =?utf-8?Q?"weird"?= name"<EMAIL_ADDRESS>(i.e. purposely containing what looks like a mime-encoded part in a name) but I can't imagine that would ever be an issue... there may be other things affected too because of how the parsers are built, it would have to be investigated.
If you're able to sanitize for exceptions you know of like that, I think that would be the way to go at least for now... we can leave this open and look when there's time or if it's affecting more people.  You could also try emailing the folks at Amway to tell them there's an issue with their emails :)  maybe they're using a house-built system that needs to be patched, or maybe it's a huge commercial system that means handling this scenario should be prioritized.

by the way you did an excellent job with this library! About 9995 out of 10000 emails can be parsed on average from my web mail client (backed by your library) without any issues.

Excellent, very happy to hear that!


K3S Import : Should not be allowed to save invalid values (0/negative values ) for control and master concurrency.

Rancher server version - 25f16c370
Currently we are allowed to enter invalid values (0/negative values ) for control and master concurrency and these incorrect values gets saved in the  cluster's  "k3supgradeStrategy".
Expected behavior:
API should error out in this case.
Also editing cluster version after this does not result in upgrade to be triggered .

Will test once the types changes to  be included  in rancher repo.

looks like types got updated. moving over

Tested with latest build from master - e78e43fea
API fails with following error message - "Validation failed in API: k3sConfig k3supgradeStrategy=InvalidFormat 422: serverConcurrency=MinLimitExceeded 422:


TabList

Hello,
how can i create own tablist on mcprotocollib proxy when someone look at TAB he will see "asdasdasdasdasd" and no one more ?
Or How can i download server tablist and add my words on that ?
@Mareckoo01 @phase @Steveice10 @109C

If you want to store TAB data from server look at #242
if you want to send TAB data, you need to send ServerPlayerListEntryPacket to connected player(s), it would look something like this
PlayerListEntry ple[] = {
		new PlayerListEntry(new GameProfile(UUID.randomUUID(), "T1"), GameMode.SURVIVAL, 0, Message.fromString("Some text")),
		new PlayerListEntry(new GameProfile(UUID.randomUUID(), "T2"), GameMode.SURVIVAL, 0, Message.fromString("Some text with §2color")),
		new PlayerListEntry(new GameProfile(UUID.randomUUID(), "T3"), GameMode.SURVIVAL, 0, Message.fromString("Some text with §bcolor"))};
ServerPlayerListEntryPacket packet = new ServerPlayerListEntryPacket(PlayerListEntryAction.ADD_PLAYER, ple);
for(Session connected : connectedClients){
	connected.send(packet);
}

@MALW3R3 cache the servers tablist


[BUG] 1st argument: not an already existing atom

Versions Used
Kaffy: 0.9.2
Phoenix: 1.6.15
Elixir: 1.14.1
What's actually happening?
I just added kaffy to a small project of mine. When I start everything, it works fine,but when I stop the app and start it again with mix phx.server a reload of the page crashes with:
Request: GET /admin/engagements/engagement/571e450d-7a85-40d2-aafe-b22d90a8d4b4
** (exit) an exception was raised:
    ** (ArgumentError) errors were found at the given arguments:

  * 1st argument: not an already existing atom

        :erlang.binary_to_existing_atom("engagements", :utf8)
        (kaffy 0.9.4) lib/kaffy/utils.ex:365: Kaffy.Utils.convert_to_atom/1
        (kaffy 0.9.4) lib/kaffy/utils.ex:361: Kaffy.Utils.convert_to_atoms/2
        (kaffy 0.9.4) lib/kaffy/utils.ex:218: Kaffy.Utils.get_resource/3
        (kaffy 0.9.4) lib/kaffy_web/controllers/resource_controller.ex:92: KaffyWeb.ResourceController.show/2
        (kaffy 0.9.4) lib/kaffy_web/controllers/resource_controller.ex:1: KaffyWeb.ResourceController.action/2
        (kaffy 0.9.4) lib/kaffy_web/controllers/resource_controller.ex:1: KaffyWeb.ResourceController.phoenix_controller_pipeline/2
        (phoenix 1.6.15) lib/phoenix/router.ex:354: Phoenix.Router.__call__/2

What should happen instead?
Not crash when reloading page after server was restarted

Does this still happen with Kaffy 0.9.4?

Yes. Below a stacktrace from 0.9.4
ArgumentError: errors were found at the given arguments:

  * 1st argument: not an already existing atom

  Module "erlang", in :erlang.binary_to_existing_atom/2
  File "lib/kaffy/utils.ex", line 365, in Kaffy.Utils.convert_to_atom/1
  File "lib/kaffy/utils.ex", line 361, in Kaffy.Utils.convert_to_atoms/2
  File "lib/kaffy/utils.ex", line 218, in Kaffy.Utils.get_resource/3
  File "lib/kaffy_web/controllers/resource_controller.ex", line 55, in KaffyWeb.ResourceController.index/2
  File "lib/kaffy_web/controllers/resource_controller.ex", line 1, in KaffyWeb.ResourceController.action/2
  File "lib/kaffy_web/controllers/resource_controller.ex", line 1, in KaffyWeb.ResourceController.phoenix_controller_pipeline/2
  File "lib/phoenix/router.ex", line 354, in Phoenix.Router.__call__/2

Yes, can confirm. This also happens in 0.9.4

We also encounter this in production. It appears that if a server has had the Kaffy homepage/initial dashboard requested, all subsequent requests to that server will serve correctly. For servers that have not yet had the initial dashboard requested, but where a direct link to e.g. a resource index is used by a user (from elsewhere), they'll encounter this error.
In practice running across multiple hosts, this means that a user could load the dashboard successfully, click to visit a resource (which then is routed to a different host) and encounter this issue.

I'll make sure this is fixed with the next release v0.9.5
Figured what the problem is so the fix just needs to be nicely implemented.

I'm also running into this issue. @aesmail can you point out what the problem is? Someone else might be able to implement a fix.

Thanks @aesmail.
I saw some "to existing atom" changes on the 0.9.5 branch, but I was still seeing errors when restarting the server and then visiting a deep link without first going to the dashboard. So I assume you're not quite done yet.

@ghenry #261 might fix the issue, but I'm worried about the use of String.to_atom/1. This might crash the app during runtime due to dynamically creating atoms. It might not be a big deal, but you never know. I could be a bad guy and try to hit random non-existent URLs just to crash the app.

For example, what do projects like https://github.com/plausible/analytics do or other projects we could look at? I'm sure it's a solved problem already?


bullet micro chart or harvey ball = update issue in element - syntax error invalid character

Hello
I was using SDK since a couple of months but it seems that some components have issues after doing an update.
For instance on one VM, i use the version <IP_ADDRESS>609070923 and everything is fine
On another VM, i use the version   <IP_ADDRESS>702071429 and  i have the following issue when trying to use these components during development :
Update issue in Element org.scn.community.databound.BulletMicroChart#BULLETMICROCHART_T1_control
SyntaxError: Invalid character
On both VMs i'm on Design Studio 1.6 SP3

Hi caznadien,
There were some namespaces changes on the library, which caused the issue. Did you check the latest repository branch?
Kind regards
Martin


Bugfix: drop calibration handle on driver drop; more control on the r…

Turns out, the "new" oneshot ADC driver does not deallocate the adc_cali_handle on drop.
I also noticed that the calibration bool from the Config does not have any effect. The driver always uses calibration, even if calibration is set to false (which it is by default).

The fix also introduces a breaking change, in that the user is now capable of selecting the calibration scheme when configuring the driver (or no calibration), with:

Calibration::None
Calibration::CurveFitting
Calibration::LineFitting

Besides the small c2 thing, i am a bit skeptical about the correctness of the magic numbers we set on the attenuation matcher. E.g the return values we define as get_max_mv.
But since that is not something you touched i am fine with the PR.

The magic numbers are taken from the 4.4.X oneshot driver documentation here.
Weird, but while the new oneshot driver does explicitly list the same raw-level-to-mV conversion formula, it does not list reference min and max mV for each attenuation. Maybe just an omission in the documentation? Also the listed reference numbers are for esp32 and esp32s2, and no info for new MCUs, but this is the best we have.
But regardless - my point is - if you find "less magical" reference numbers for doing raw-to-mV conversion without calibration (and yes, I still think it is worth it to support this use case given that its conversion formula is explicitly documented in ESP IDF - both in legacy and new oneshot driver) - please paste the link here and I would gladly use your reference numbers instead! :D


character(13) data type in PostgreSQL database scaffolded as char data type in .NET Core 2.0

I'm using Npgsql.EntityFrameworkCore.PostgreSQL version 2.0.1
I have the next issue. pgAdmin 4 generates SQL as following:
rfc character(13) COLLATE pg_catalog."default" NOT NULL,
CONSTRAINT pk_empresas PRIMARY KEY (rfc)

The entire database is scaffolded into my .NET Core 2.0 project using the next command in Package Manager Console, which is done correctly.
Scaffold-DbContext "Host=localhost;Database=database;Username=pgadmin;Password=xxxx" Npgsql.EntityFrameworkCore.PostgreSQL -force
Until here, everything is fine, but when I go the scaffolded class/model of the table where I have the SQL generated by pgAdmin 4 as commented before.
I have the property scaffolded as following:
public char Rfc { get; set; }
And I can't really work with this, as this only holds a single UTF-16 character :sob:
I need to explicitly work with a set of characters with a length of 13.
Is there something I can do at the moment?

Original SO: https://stackoverflow.com/questions/49948827/npgsql-data-type-mapping-of-character-from-postgresql-to-net-core-2-0?noredirect=1#comment86940421_49948827


We need to use a .csv file of NuGet packages in order for the package information to be displayed on the API docs pages.

We need to use a .csv file of NuGet packages in order for the package information to be displayed on the API docs pages.
See https://ceapex.visualstudio.com/Engineering/_workitems/edit/654750.
Instructions are at https://microsoft.sharepoint.com/teams/DotNETandWebDevContent/_layouts/15/Doc.aspx?sourcedoc={0a811e54-964b-4322-83c0-52530ce1ba85}&action=edit&wd=target(API generation process.one|8127a7eb-32d2-4027-b369-951d4527137a%2FAPI docs generation process|7b904331-cd68-4e7f-8605-504cbbac7944%2F).

Associated WorkItem - 99252
Originally posted by @gewarren in https://github.com/dotnet/dotnet-api-docs/issues/9060

@Toppole6 can you give more context on this issue and why it is in the azure-sdk repo?


AVX512 CMake target

Adds a new AVX512 target for processors supporting only the core AVX512 functionality.

Resolved conflicts with master.


Is there any ability to provide power to OBD pins via the openxc standard?

Is there any ability in the openxc firmware specification to provide power to other IO pins?
Eg provide 5-10v on a separate OBD pin?
This is something that is specified in J2534, however J2534 is a windows only standard meaning to port J2534 functionality to Android etc you must rewrite each windows driver.

No, this isn't supported easily. We do use some GPIO. However, none of them are routed to the OBD-II pins. Also, would have to check what the current limit would be and it would need to be level shifted to get above 5V.
If you want to dig deeper, here is all the MCU config:
https://github.com/openxc/vi-firmware/tree/master/src/platform/lpc17xx
And here is the schematic. You would need to add a connection to the OBD-II pins:
http://vi.openxcplatform.com/electrical/sources/index.html
I will close this issue.


xlxb to csv with negative number

I try to convert a xlsb to csv. The xls contains one cell with color red and a negative number, after the conversion the cell is "--1485.85 ". There is two minus symbols.
The file is here => http://www.filedropper.com/rednegnumber
My code for the convertion :
var f = document.getElementById('input').files[0];
var reader = new FileReader();
var name = f.name;
reader.onload = function(e) {
  var data = e.target.result;

  var workbook = XLSX.read(data, {type: 'binary'});

  var sheet1 = workbook.Sheets[workbook.SheetNames[0]];
  console.debug(XLSX.utils.sheet_to_csv(sheet1,{"FS":";"}));
};
reader.readAsBinaryString(f);

So It's a bug ?

The positive numbers are black and the negative numbers are red (in excel).
The format code of the cell is : # ##0,00;[RED]-# ##0,00
Function eval_fmt(fmt, v, opts, flen) is called with parameters :

fmt = "[RED]-# ##0,00"
v = -3419.19

This function return -TheNumber, so --3419.19
I update the case "["
case "[":
                    o = c;
                    while (fmt[i++] !== "]" && i < fmt.length)
                        o += fmt[i];
                    if (o.substr(-1) !== "]")
                        throw 'unterminated "[" block: |' + o + "|";
                    if (fmt[i] === "\\" && fmt[i+1] === "-") {
                        o += fmt[i];
                        o += fmt[i++];
                        i++;
                    }
                    if (o.match(abstime)) {
                        if (dt == null) {
                            dt = parse_date_code(v, opts);
                            if (dt == null)
                                return ""
                        }
                        out[out.length] = {
                            t : "Z",
                            v : o.toLowerCase()
                        }
                    } else {
                        o = ""
                    }
                    break;

And it's work, but I don't know if it's correct, it's maybe before the function eval_fmt that the fixed must be applied.

@bertrand-sifre We addressed the negative value in an earlier commit.  The correct format based on our efforts to reproduce is
#,##0.00;[RED]-#,##0.00

Excel generally generates the file with the US delimiters (comma for thousands, period for decimal) and uses locale settings to determine the separators.  Can you share a file with that specific format?

Ok. It was two years ago, I don't have the file with the probleme, sorry.

No worries, I think the confusion probably stemmed fro a difference between the excel interface and the actual number format (it will show one thing but actually store something different in the file)


Package vadimdemedes/ink not compatible with React 19

There is a current issue with ink that is breaking support for React 19 (affects Next 15): vadimdemedes/ink#688
Discussion: https://discord.com/channels/983865673656705025/1318230919579963473

Hi, I found myself in the same situation so I decided to fork the ink project to upgrade to react 19 and the latest version of react-reconciler in the meantime. I then override the react version of sst + the ink package.
"overrides": {
    "sst": {
      "ink": "https://github.com/QuentinFontenay/ink/tree/react-v19.0.0",
      "react": "$react"
    }
  },


API

what will the basic API look like for this package?
will it be class based or functional?

@untzag I think we should use a class based API.


Resilient USP info handler

In the process of operation, we found a situation where Oktopus stops functioning when receiving an incorrect USP message from an agent, namely:
GET_RESP on a GET Device.Deviceinfo request.
The error causes the adapter container to crash with the error:
2024/07/22 08:39:58 info.go:15: Device proto::puma7-00:50:f1:20:00:53 info, mtp: mqtt
panic: runtime error: index out of range [0] with length 0
goroutine 55 [running]:
[github.com/OktopUSP/oktopus/backend/services/mtp/adapter/internal/events/handler.parseDeviceInfoMsg({0xc0000f042c](http://github.com/OktopUSP/oktopus/backend/services/mtp/adapter/internal/events/handler.parseDeviceInfoMsg({0xc0000f042c), 0x1e}, {0x40e7fb?, 0xc2490a?}, {0xc0004c0000, 0x2e8})
/app/internal/events/handler/info.go:52 +0x465
[github.com/OktopUSP/oktopus/backend/services/mtp/adapter/internal/events/handler.(*Handler).HandleDeviceInfo(0xc0000ca1e0](http://github.com/OktopUSP/oktopus/backend/services/mtp/adapter/internal/events/handler.(*Handler).HandleDeviceInfo(0xc0000ca1e0), {0xc0000f042c, 0x1e}, {0xc0000f0420, 0x2f}, {0xc0004c0000, 0x2e8} , {0xa2650c, 0x4}, ...)
/app/internal/events/handler/info.go:16 +0x20e
github.com/OktopUSP/oktopus/backend/services/mtp/adapter/internal/events.StartEventsListener.func1()
/app/internal/events/events.go:55 +0x486
created by github.com/OktopUSP/oktopus/backend/services/mtp/adapter/internal/events.StartEventsListener in goroutine 1
/app/internal/events/events.go:26 +0x1e5
We found the following explanation: in the module oktopus/backend/services/mtp/adapter/internal/events/handler/info.go (and in the new version it is oktopus/backend/services/mtp/adapter/internal/events/usp_handler
/info.go) an error occurs in the parsing process
device.Vendor = msg.ReqPathResults[0].ResolvedPathResults[0].ResultParams[“Manufacturer”]
device.Model = msg.ReqPathResults[1].ResolvedPathResults[0].ResultParams[“ModelName”]
device.Version = msg.ReqPathResults[2].ResolvedPathResults[0].ResultParams[“SoftwareVersion”]
device.ProductClass = msg.ReqPathResults[4].ResolvedPathResults[0].ResultParams[“ProductClass”]
Since the object msg.ReqPathResults[0].ResolvedPathResults[0] takes the value nil
In our opinion, it would be useful to add handling of such an exception to the code, as it would protect the controller and other users from incorrect behavior of one of the agents.
If necessary, we can try to provide more detailed information.
Sincerely, Oleksandr.


[BEAM-1634] Flink: register known IOChannelFactories

R: @aljoscha

Coverage decreased (-0.01%) to 70.041% when pulling 971335e4d85982b697e30a6ba331eabc7b38f3a0 on davorbonaci:flink-gcs-fix into 1fd52f53c92ece265ed41bd7f2a06b0cdf6f8afd on apache:master.

LGTM!
Your concern is valid but we lately see more and more people moving to a one-job-per-cluster setting on top of YARN or Mesos. (Which is also what we usually suggest people do.)


How protect server ip form getting banned by opera

Dear support.
I want to connect to opera proxy via TOR proxy, I have rotated tor proxy.
How can I connect to opera-proxy and pass my TOR proxy to it?
export http_proxy=[tor_proxy] ./opera-proxy does not work.
Reards

Hi!
Assuming your tor instance accepts SOCKS5 connections on port 9050, you can run it like this:
opera-proxy -proxy socks5h://<IP_ADDRESS>:9050 -timeout 30s

Extended timeout parameter is set to 30 seconds (instead of default 10 seconds) for practical reasons because Tor can be sluggish.


Zxing crash on iOS 10

Hi,
we are using the latest stable version of Zxing (2.1.47) and it crashes on iOS 10.0.1.
Are you aware of this? Any fix on the road?
Thank you

Same for me.
Xamarin Forms <IP_ADDRESS>
ZXing.Net.Mobile(.Forms) 2.1.47

2016-09-14 17:29:59.033 xxxxxxxxxxxxxx.iOS[336:35119] ZXingScannerView.Setup() took 0.128 ms.
2016-09-14 17:29:59.033 xxxxxxxxxxxxxx.iOS[336:35119] StartScanning
2016-09-14 17:29:59.096 xxxxxxxxxxxxxx.iOS[336:35119] PERF: Alloc AVCaptureVideoPreviewLayer took 7.633 ms.
then it crashes...

Ok, I found the solution.
Just add the NSCameraUsageDescription key to info.plist
<key>NSCameraUsageDescription</key>
<string>Can we use your camera</string>

Works. Thanks @iMacX

Unfortunately, that fix doesn't seem to work for iOS apps. I also tried it by creating a new blank single view iOS app but it crash on that as well. With or without the NSCameraUsageDescription in plist.info.
Shortened stacktrace:

critical:   at  <0xffffffff>
critical:   at (wrapper managed-to-native) ObjCRuntime.Messaging.void_objc_msgSend_IntPtr (intptr,intptr,intptr) <0x00012>
critical:   at AVFoundation.AVCaptureSession.set_SessionPreset (Foundation.NSString) [0x00022] in /Users/builder/data/lanes/3426/6c3fee4d/source/xamarin-macios/src/build/ios/native/AVFoundation/AVCaptureSession.g.cs:418
critical:   at ZXing.Mobile.ZXingScannerView.SetupCaptureSession () <IL 0x000f9, 0x00a0a>
critical:   at ZXing.Mobile.ZXingScannerView.b__36_0 () <IL 0x00002, 0x00063>
critical:   at Foundation.NSActionDispatcher.Apply () [0x00000] in /Users/builder/data/lanes/3426/6c3fee4d/source/xamarin-macios/src/Foundation/NSAction.cs:57
critical:   at (wrapper runtime-invoke) object.runtime_invoke_void__this__ (object,intptr,intptr,intptr) <IL 0x00020, 0x000cc>

lokan82, do you think this problem might be related to https://github.com/Redth/ZXing.Net.Mobile/issues/422

Thanks @iMacX

the fix doesnt work for me either

This fixed for me, for god sake Xamarin, hire a QA team... I like your product but I can't dev 10min without stuff like that constantly happening :(

@NicolasDorier This bug has nothing to do with xamarin. It's a new requirement in iOS 10 introduced by Apple.

@iMacX so maybe at least a nice exception instead of a saddistic segfault error, would save some man-hours.

The fix from @iMacX doesn't work for me either.
Xamarin.iOS <IP_ADDRESS>
2016-10-04 13:51:56.208 xxx.ios[32988:932942] Starting to scan...
2016-10-04 13:51:56.229 xxx.ios[32988:932942] ZXingScannerView.Setup() took 13.191 ms.
2016-10-04 13:51:56.229 xxx.ios[32988:932942] StartScanning
2016-10-04 13:51:56.238 xxx.ios[32988:932942] critical: Stacktrace:
2016-10-04 13:51:56.239 xxx.ios[32988:932942] critical:   at <unknown> <0xffffffff>
2016-10-04 13:51:56.239 xxx.ios[32988:932942] critical:   at (wrapper managed-to-native) ObjCRuntime.Messaging.void_objc_msgSend_IntPtr (intptr,intptr,intptr) <0x00012>
2016-10-04 13:51:56.240 xxx.ios[32988:932942] critical:   at AVFoundation.AVCaptureSession.set_SessionPreset (Foundation.NSString) [0x0001a] in /Users/builder/data/lanes/3426/6c3fee4d/source/xamarin-macios/src/build/ios/native/AVFoundation/AVCaptureSession.g.cs:418
2016-10-04 13:51:56.240 xxx.ios[32988:932942] critical:   at ZXing.Mobile.ZXingScannerView.SetupCaptureSession () <IL 0x000f9, 0x00a0e>
2016-10-04 13:51:56.240 xxx.ios[32988:932942] critical:   at ZXing.Mobile.ZXingScannerView.<StartScanning>b__36_0 () <IL 0x00002, 0x00063>
2016-10-04 13:51:56.241 xxx.ios[32988:932942] critical:   at Foundation.NSActionDispatcher.Apply () [0x00000] in /Users/builder/data/lanes/3426/6c3fee4d/source/xamarin-macios/src/Foundation/NSAction.cs:57
2016-10-04 13:51:56.241 xxx.ios[32988:932942] critical:   at (wrapper runtime-invoke) object.runtime_invoke_void__this__ (object,intptr,intptr,intptr) <IL 0x00020, 0x000cc>
2016-10-04 13:51:56.241 xxx.ios[32988:932942] critical:   at <unknown> <0xffffffff>
2016-10-04 13:51:56.242 xxx.ios[32988:932942] critical:   at (wrapper managed-to-native) ObjCRuntime.Messaging.void_objc_msgSend_IntPtr_IntPtr_bool (intptr,intptr,intptr,intptr,bool) <0x00012>
2016-10-04 13:51:56.242 xxx.ios[32988:932942] critical:   at Foundation.NSObject.InvokeOnMainThread (System.Action) [0x00007] in /Users/builder/data/lanes/3426/6c3fee4d/source/xamarin-macios/src/Foundation/NSObject2.cs:560
2016-10-04 13:51:56.243 xxx.ios[32988:932942] critical:   at ZXing.Mobile.ZXingScannerView.StartScanning (System.Action`1<ZXing.Result>,ZXing.Mobile.MobileBarcodeScanningOptions) <IL 0x00060, 0x00297>
2016-10-04 13:51:56.243 xxx.ios[32988:932942] critical:   at ZXing.Mobile.ZXingScannerViewController.<ViewDidAppear>b__33_1 () <IL 0x00018, 0x000e7>
2016-10-04 13:51:56.243 xxx.ios[32988:932942] critical:   at Foundation.NSAsyncActionDispatcher.Apply () [0x00000] in /Users/builder/data/lanes/3426/6c3fee4d/source/xamarin-macios/src/Foundation/NSAction.cs:163
2016-10-04 13:51:56.243 xxx.ios[32988:932942] critical:   at (wrapper runtime-invoke) object.runtime_invoke_void__this__ (object,intptr,intptr,intptr) <IL 0x00020, 0x000cc>
2016-10-04 13:51:56.244 xxx.ios[32988:932942] critical:   at <unknown> <0xffffffff>
2016-10-04 13:51:56.244 xxx.ios[32988:932942] critical:   at (wrapper managed-to-native) UIKit.UIApplication.UIApplicationMain (int,string[],intptr,intptr) <0x00012>
2016-10-04 13:51:56.244 xxx.ios[32988:932942] critical:   at UIKit.UIApplication.Main (string[],intptr,intptr) [0x00005] in /Users/builder/data/lanes/3426/6c3fee4d/source/xamarin-macios/src/UIKit/UIApplication.cs:79
2016-10-04 13:51:56.245 xxx.ios[32988:932942] critical:   at UIKit.UIApplication.Main (string[],string,string) [0x00038] in /Users/builder/data/lanes/3426/6c3fee4d/source/xamarin-macios/src/UIKit/UIApplication.cs:63
2016-10-04 13:51:56.245 xxx.ios[32988:932942] critical:   at xxx.ios.ios.Application.Main (string[]) [0x00002] in C:\Path\To\Solution\xxx.ios.ios\Main.cs:15
2016-10-04 13:51:56.245 xxx.ios[32988:932942] critical:   at (wrapper runtime-invoke) <Module>.runtime_invoke_void_object (object,intptr,intptr,intptr) <IL 0x00051, 0x001d6>
2016-10-04 13:51:56.246 xxx.ios[32988:932942] critical:
Native stacktrace:
2016-10-04 13:51:56.246 xxx.ios[32988:932942] critical:
=================================================================
Got a SIGSEGV while executing native code. This usually indicates
a fatal error in the mono runtime or one of the native libraries
used by your application.
=================================================================

I found that it works when debugging on a real device, when I use the simulator it dies with the error above.

@lokan82 we're getting the same crash on real devices. Are you seeing it work on real devices with or without the NSCameraUsageDescription key?

With NSCameraUsageDescription added to info.plist

@iMacX and @lokan82 thanks for the tip, the NSCameraUsageDescription fix worked for me on a real device (but not on the simulator).

Got exactly the same issue.
Does a fix is on the road?
Regards,
Estema

Even with NSCameraUsageDescription added to info.plist it's not working for me.

@NicolasDorier Apple decided to cause a segfault in iOS 10 if you are missing the info.plist entry, this isn't xamarin's fault. Furthermore this Lib is not officially supported by Xamarin, I just happen to work at Xamarin and work on it in my spare time. So apologies if this free and open source project isn't quite perfect. You are always welcome to submit a PR with any fixes and improvements even if it's just updating the README. Cheers!

@Redth sorry, I was not even using ZXing when I encountered this exception. Somehow, I thought I was commenting on the Xamarin project. This problem can't be fixed at ZXing level, it is the fault of Xamarin.

On iOS 10, I've made it worked by adding NSCameraUsageDescription in plist and by asking permission for camera usage with this kind of code :
`              void BtnScan_TouchUpInside(object sender, EventArgs e)
{
switch (AVCaptureDevice.GetAuthorizationStatus(AVMediaType.Video))
{
case AVAuthorizationStatus.Authorized:
// User granted camera access.
scan();
break;
			case AVAuthorizationStatus.Denied:
				// User denied camera access.
				break;

			case AVAuthorizationStatus.Restricted:
				// Camera access is restricted. Should not happen for AVMediaType.Video.
				// Check AVCaptureDevice.Devices.Length > 0 to determine if there is a video device
				// before using this code.
				break;

			case AVAuthorizationStatus.NotDetermined:
				// Ask user for camera access.

				if (AVCaptureDevice.RequestAccessForMediaTypeAsync(AVMediaType.Video).Result)
				{
					// User granted camera access.
					scan();
				}

				else
				{
					// User denied camera access.
				}
				break;

			default:
				break;
		}

	}

	void scan()
	{
		//Create a new instance of our scanner
		MobileBarcodeScanner scanner;
		scanner = new MobileBarcodeScanner(this);

		Task.Run(async () =>
	   {
		   //Tell our scanner to use the default overlay
		   scanner.UseCustomOverlay = false;
		   //We can customize the top and bottom text of the default overlay
		   //Start scanning
		   var result = await scanner.Scan();

		   HandleScanResult(result);
	   });
	}

	void HandleScanResult(ZXing.Result result)
	{
		string msg = "";

		if (result != null && !string.IsNullOrEmpty(result.Text))
			msg = "Found Barcode: " + result.Text;
		else
			msg = "Scanning Canceled!";

		this.InvokeOnMainThread(() =>
		{
			var av = new UIAlertView("Barcode Result", msg, null, "OK", null);
			av.Show();
		});
	}`

I hope this will help you.

I could fixed the problem.
Error will occur in the next source.
ZXing.Net.Mobile.iOS.ZXingScannerView.cs
bool SetupCaptureSession ()
my source is below.
http://itblogdsi.blog.fc2.com/blog-entry-71.html

I haven't encountered this issue because I don't have an iOS 10 device. I was thinking about upgrading my iOS 6S, but I hate applying Apple updates.  I'm currently using Zxing.Net.Mobile -Version <IP_ADDRESS> and Zxing.Net.Mobile.Forms -Version <IP_ADDRESS> in my XF iOS app and it works great. I am also running Xamarin.iOS <IP_ADDRESS>. Do I need to add NSCameraUsageDescription to info.plist in order for my app to work with iOS 10? Do I need to do anything else? Thanks in advance for the help and clarification.

Add the new entry to the info.plist worked for me.
Thank you.

For those experiencing a crash while using the simulator, this appears to be due to a bug in iOS 10 itself.

Project worked after adding to Info.plist Property::"Privacy - Camera Usage Description" // Value::"Taking picture barcode".

Just sharing for someone novice like me....my key worked but only after cleaning the solution

I am glad that I found this post and it did indeed resolve the problem for me too.
I don't use ZXing, but am decoding bar codes using the standard IOS bar code decoding.
Having said that: can anyone explain what the root cause of this is ? How can it be prevented in the future ... it seems like a coding mistake in the underlying infrastructure, someone must have added this description and forgot to check whether the string was initialized before using it. So a null pointer was probably encountered (just guessing here), was it Apple or Xamarin ?
The stack traces are not particularly useful, but I guess you need debug versions of the libraries to make that more descriptive.
BTW do we need to include the string in Localizable strings to make it readable for non-English users ?

I am have IOS 9.3.5 installed and it works fine when i am debugging.   When I run the app normal it crashes the application after the barcode is scanned.  Any ideas why?
ZXing.Net.Mobile = 2.2.9
ZXing.Net.Mobile.Forms = 2.2.9
Xamarin.Forms = <IP_ADDRESS>

Thanks for reporting this issue!  Unforunately it took me way too long to respond 😭.  Sorry, I apologize!  Recently the source code for this project was completely refactored to modernize it.  Many PR's were included in this effort, and many bugs were hopefully fixed.  Please try out the latest 3.x series of NuGet packages (currently in prerelease).  To try and make the project more maintainable in my spare time going forward, I've decided to close all existing issues to start with a clean slate.  If you're still experiencing this issue on the newest version, please open a new issue with as much detail as possible.  Thank you for your patience and understanding!  Happy scanning!

Thanks for reporting this issue! Unforunately it took me way too long to respond 😭. Sorry, I apologize! Recently the source code for this project was completely refactored to modernize it. Many PR's were included in this effort, and many bugs were hopefully fixed. Please try out the latest 3.x series of NuGet packages (currently in prerelease). To try and make the project more maintainable in my spare time going forward, I've decided to close all existing issues to start with a clean slate. If you're still experiencing this issue on the newest version, please open a new issue with as much detail as possible. Thank you for your patience and understanding! Happy scanning!

Hi Redth,
We have the need to include the scanner into our mobile app and one of my developers suggested to use ZXing.Net.Mobile scanner package.  I have follow your example and suggestions from others and still haven't been able to get it to scan.  I am using the latest version of xamarin forms and ZXing.Net.Mobile package.   Do you have a workable version /example that I can try and see if I can spot what I did wrong?  Any help is greatly appreciated.
Thanks,
Daniel


renter can now provide more thorough file status

This ended up requiring a fair amount of changes. I also realized that the renter was never calling save or load.
One thing that I left unfinished was that the renter's file is never set. This may result in errors (perhaps silent) while the renter is running.

why didn't the build run? How do we get the build to run?

As long as the build passes, the problems I noticed should be fixed.


string contains null byte

Hey there,
I'm getting this error when trying to save my model: string contains null byte
Here's the stack trace from the console:
ArgumentError: string contains null byte
        from /usr/local/bundle/gems/activerecord-4.1.6/lib/active_record/connection_adapters/postgresql/oid.rb:30:in `unescape_bytea'
        from /usr/local/bundle/gems/activerecord-4.1.6/lib/active_record/connection_adapters/postgresql/oid.rb:30:in `type_cast'
        from /usr/local/bundle/gems/activerecord-4.1.6/lib/active_record/connection_adapters/postgresql_adapter.rb:156:in `type_cast'
        from /usr/local/bundle/gems/activerecord-4.1.6/lib/active_record/attribute_methods/dirty.rb:102:in `_field_changed?'
        from /usr/local/bundle/gems/activerecord-4.1.6/lib/active_record/attribute_methods/dirty.rb:78:in `save_changed_attribute'
        from /usr/local/bundle/gems/activerecord-4.1.6/lib/active_record/attribute_methods/dirty.rb:66:in `write_attribute'
        from /usr/local/bundle/gems/activerecord-4.1.6/lib/active_record/attribute_methods.rb:395:in `[]='
        from /usr/local/bundle/gems/strongbox-0.7.0/lib/strongbox/lock.rb:50:in `encrypt'
        from /usr/local/bundle/gems/strongbox-0.7.0/lib/strongbox.rb:80:in `block in encrypt_with_public_key'
        from (irb):2
        from /usr/local/bundle/gems/railties-4.1.6/lib/rails/commands/console.rb:90:in `start'
        from /usr/local/bundle/gems/railties-4.1.6/lib/rails/commands/console.rb:9:in `start'
        from /usr/local/bundle/gems/railties-4.1.6/lib/rails/commands/commands_tasks.rb:69:in `console'
        from /usr/local/bundle/gems/railties-4.1.6/lib/rails/commands/commands_tasks.rb:40:in `run_command!'
        from /usr/local/bundle/gems/railties-4.1.6/lib/rails/commands.rb:17:in `<top (required)>'
        from /usr/local/bundle/gems/activesupport-4.1.6/lib/active_support/dependencies.rb:247:in `require'
... 4 levels...
        from /usr/local/bundle/gems/activesupport-4.1.6/lib/active_support/dependencies.rb:241:in `load'
        from /usr/local/bundle/gems/activesupport-4.1.6/lib/active_support/dependencies.rb:241:in `block in load'
        from /usr/local/bundle/gems/activesupport-4.1.6/lib/active_support/dependencies.rb:232:in `load_dependency'
        from /usr/local/bundle/gems/activesupport-4.1.6/lib/active_support/dependencies.rb:241:in `load'
        from /usr/local/bundle/gems/spring-1.2.0/lib/spring/commands/rails.rb:6:in `call'
        from /usr/local/bundle/gems/spring-1.2.0/lib/spring/command_wrapper.rb:38:in `call'
        from /usr/local/bundle/gems/spring-1.2.0/lib/spring/application.rb:183:in `block in serve'
        from /usr/local/bundle/gems/spring-1.2.0/lib/spring/application.rb:156:in `fork'
        from /usr/local/bundle/gems/spring-1.2.0/lib/spring/application.rb:156:in `serve'
        from /usr/local/bundle/gems/spring-1.2.0/lib/spring/application.rb:131:in `block in run'
        from /usr/local/bundle/gems/spring-1.2.0/lib/spring/application.rb:125:in `loop'
        from /usr/local/bundle/gems/spring-1.2.0/lib/spring/application.rb:125:in `run'
        from /usr/local/bundle/gems/spring-1.2.0/lib/spring/application/boot.rb:18:in `<top (required)>'
        from /usr/local/lib/ruby/2.1.0/rubygems/core_ext/kernel_require.rb:55:in `require'
        from /usr/local/lib/ruby/2.1.0/rubygems/core_ext/kernel_require.rb:55:in `require'

Any ideas?

Looks like adding the :base64 option fixed the problem.  I think this may have something to do with the bytea type in postgres? Not sure if this changed between rails version because I've never had this problem before.

Looks like it's a bug in Rails when using the bytea type. https://bitbucket.org/ged/ruby-pg/issue/198/argumenterror-string-contains-null-byte Using base64:. as you found type will let you you work around it.