Some controls added in exceptions.json file are ignored and fail

Description

Hi everyone
I'm unable to make kubescape ignore some controls that I added to my exceptions.json file when scanning a deployment. I get the same result with a non-kustomized deployment.
For example, in my case, C-0211 and C-0237.
Environment
OS: Ubuntu 22.04 in WSL and Ubuntu 20.04 in Azure Container Instance (agent)
Version: 2.2.0, 2.2.3 (tried both of them)
Steps To Reproduce

Create a exceptions.json and a controls-inputs.json file, fill them (as shown in Additional context)
Use either local or remote artifacts
Execute command shown in Additional context

Expected behavior

The exceptions are ignored
Actual Behavior

Theese two particular exceptions are not ignored:

CIS-5.7.3 Apply Security Context to Your Pods and ...
Check if signature exists

Controls: 211 (Failed: 8, Passed: 197, Action Required: 6)
Failed Resources by Severity: Critical — 0, High — 2, Medium — 4, Low — 2

+----------+-------------------------------------------------------+------------------+---------------+-------------------+
| SEVERITY |                     CONTROL NAME                      | FAILED RESOURCES | ALL RESOURCES |   % RISK-SCORE    |
+----------+-------------------------------------------------------+------------------+---------------+-------------------+
| High     | Forbidden Container Registries                        |        0         |       1       | Action Required * |
| High     | Resources memory limit and request                    |        0         |       1       | Action Required * |
| High     | Applications credentials in configuration files       |        0         |       1       | Action Required * |
| High     | Resources CPU limit and request                       |        0         |       1       | Action Required * |
| High     | CIS-5.7.3 Apply Security Context to Your Pods and ... |        1         |       1       |       100%        |
| High     | Verify image signature                                |        0         |       1       | Action Required * |
| High     | Check if signature exists                             |        1         |       1       |       100%        |
| Medium   | Ingress and Egress blocked                            |        1         |       1       |       100%        |
| Medium   | Configured liveness probe                             |        1         |       1       |       100%        |
| Medium   | CIS-5.1.4 Minimize Container Registries to only th... |        0         |       1       | Action Required * |
| Medium   | CIS-5.4.1 Prefer using secrets as files over secre... |        1         |       1       |       100%        |
| Medium   | CIS-5.7.2 Ensure that the seccomp profile is set t... |        1         |       1       |       100%        |
| Low      | Label usage for resources                             |        1         |       1       |       100%        |
| Low      | K8s common labels usage                               |        1         |       1       |       100%        |
+----------+-------------------------------------------------------+------------------+---------------+-------------------+
|          |                   RESOURCE SUMMARY                    |        1         |       2       |      17.87%       |
+----------+-------------------------------------------------------+------------------+---------------+-------------------+
FRAMEWORKS: AllControls (risk: 8.86), NSA (risk: 7.59), MITRE (risk: 0.00), AllControls (risk: 8.86), ArmoBest (risk: 11.11), cis-eks-t1.2.0 (risk: 14.29), cis-v1.23-t1.0.1 (risk: 55.17), DevOpsBest (risk: 19.51), MITRE (risk: 0.00), NSA (risk: 7.59)

* Control configurations are empty
[success] Scan results saved. filename: report.xml

Overall risk-score (0- Excellent, 100- All failed): 18

Additional context

Executed command:
kustomize build . | kubescape scan - --format junit --exclude-namespaces kube-system,kube-public --use-artifacts-from ./artifacts/ --exceptions ./exceptions.json --fail-threshold 40 --severity-threshold high --controls-config ./controls-inputs.json

Exceptions file:
[
    {
    "name": "my-framework-exceptions",
    "policyType": "postureExceptionPolicy",
        "actions": [
            "alertOnly"
        ],
    "resources": [
        {
            "designatorType": "Attributes",
            "attributes": {
                "labels": ".*"
            }
        }
    ],
    "posturePolicies": [
        {
            "frameworkName": "cis-eks-t1.2.0"
        }
    ]
    },
    {
        "name": "my-framework-exception-0004",
        "policyType": "postureExceptionPolicy",
        "actions": [
            "alertOnly"
        ],
        "resources": [
                {
                    "designatorType": "Attributes",
                    "attributes": {
                        "labels": ".*"
                    }
                }
        ],
        "posturePolicies": [
            {
                "controlID": "C-0004"
            }
        ]
    },
    {
        "name": "my-framework-exception-0012",
        "policyType": "postureExceptionPolicy",
        "actions": [
            "alertOnly"
        ],
        "resources": [
                {
                    "designatorType": "Attributes",
                    "attributes": {
                        "labels": ".*"
                    }
                }
        ],
        "posturePolicies": [
            {
                "controlID": "C-0012"
            }
        ]
    },
    {
        "name": "my-framework-exception-0055",
        "policyType": "postureExceptionPolicy",
        "actions": [
            "alertOnly"
        ],
        "resources": [
                {
                    "designatorType": "Attributes",
                    "attributes": {
                        "labels": ".*"
                    }
                }
        ],
        "posturePolicies": [
            {
                "controlID": "C-0055"
            }
        ]
    },
    {
        "name": "my-framework-exception-0076",
        "policyType": "postureExceptionPolicy",
        "actions": [
            "alertOnly"
        ],
        "resources": [
                {
                    "designatorType": "Attributes",
                    "attributes": {
                        "labels": ".*"
                    }
                }
        ],
        "posturePolicies": [
            {
                "controlID": "C-0076"
            }
        ]
    },
    {
        "name": "my-framework-exception-0193",
        "policyType": "postureExceptionPolicy",
        "actions": [
            "alertOnly"
        ],
        "resources": [
                {
                    "designatorType": "Attributes",
                    "attributes": {
                        "labels": ".*"
                    }
                }
        ],
        "posturePolicies": [
            {
                "controlID": "C-0193"
            }
        ]
    },
    {
        "name": "my-framework-exception-0202",
        "policyType": "postureExceptionPolicy",
        "actions": [
            "alertOnly"
        ],
        "resources": [
                {
                    "designatorType": "Attributes",
                    "attributes": {
                        "labels": ".*"
                    }
                }
        ],
        "posturePolicies": [
            {
                "controlID": "C-0202"
            }
        ]
    },
    {
        "name": "my-framework-exception-0211",
        "policyType": "postureExceptionPolicy",
        "actions": [
            "disable"
        ],
        "resources": [
                {
                    "designatorType": "Attributes",
                    "attributes": {
                        "labels": ".*"
                    }
                }
        ],
        "posturePolicies": [
            {
                "controlID": "C-0211"
            }
        ]
    },
    {
        "name": "my-framework-exception-0225",
        "policyType": "postureExceptionPolicy",
        "actions": [
            "alertOnly"
        ],
        "resources": [
                {
                    "designatorType": "Attributes",
                    "attributes": {
                        "labels": ".*"
                    }
                }
        ],
        "posturePolicies": [
            {
                "controlID": "C-0225"
            }
        ]
    },
    {
        "name": "my-framework-exception-0236",
        "policyType": "postureExceptionPolicy",
        "actions": [
            "alertOnly"
        ],
        "resources": [
                {
                    "designatorType": "Attributes",
                    "attributes": {
                        "labels": ".*"
                    }
                }
        ],
        "posturePolicies": [
            {
                "controlID": "C-0236"
            }
        ]
    },
    {
        "name": "my-framework-exception-0237",
        "policyType": "postureExceptionPolicy",
        "actions": [
            "alertOnly"
        ],
        "resources": [
                {
                    "designatorType": "Attributes",
                    "attributes": {
                        "labels": ".*"
                    }
                }
        ],
        "posturePolicies": [
            {
                "controlID": "C-0237"
            }
        ]
    }
]

Controls file:
{
  "cpu_limit_max": [],
  "cpu_limit_min": [],
  "cpu_request_max": [],
  "cpu_request_min": [],
  "imageRepositoryAllowList": [
    ".*azurecr.io"],
  "insecureCapabilities": [
    "SETPCAP",
    "NET_ADMIN",
    "NET_RAW",
    "SYS_MODULE",
    "SYS_RAWIO",
    "SYS_PTRACE",
    "SYS_ADMIN",
    "SYS_BOOT",
    "MAC_OVERRIDE",
    "MAC_ADMIN",
    "PERFMON",
    "ALL",
    "BPF"
  ],
  "k8sRecommendedLabels": [
    "app.kubernetes.io/name",
    "app.kubernetes.io/instance",
    "app.kubernetes.io/version",
    "app.kubernetes.io/component",
    "app.kubernetes.io/part-of",
    "app.kubernetes.io/managed-by",
    "app.kubernetes.io/created-by"
  ],
  "listOfDangerousArtifacts": [
    "bin/bash",
    "sbin/sh",
    "bin/ksh",
    "bin/tcsh",
    "bin/zsh",
    "usr/bin/scsh",
    "bin/csh",
    "bin/busybox",
    "usr/bin/busybox"
  ],
  "max_critical_vulnerabilities": [
    "5"
  ],
  "max_high_vulnerabilities": [
    "10"
  ],
  "memory_limit_max": [],
  "memory_limit_min": [],
  "memory_request_max": [],
  "memory_request_min": [],
  "publicRegistries": [],
  "recommendedLabels": [
    "app",
    "tier",
    "phase",
    "version",
    "owner",
    "env"
  ],
  "sensitiveInterfaces": [
    "nifi",
    "argo-server",
    "weave-scope-app",
    "kubeflow",
    "kubernetes-dashboard",
    "jenkins",
    "prometheus-deployment"
  ],
  "sensitiveKeyNames": [
    "aws_access_key_id",
    "aws_secret_access_key",
    "azure_batchai_storage_account",
    "azure_batchai_storage_key",
    "azure_batch_account",
    "azure_batch_key",
    "secret",
    "key",
    "password",
    "pwd",
    "token",
    "jwt",
    "bearer",
    "credential"
  ],
  "sensitiveValues": [
    "BEGIN \\w+ PRIVATE KEY",
    "PRIVATE KEY",
    "eyJhbGciO",
    "JWT",
    "Bearer",
    "_key_",
    "_secret_"
  ],
  "sensitiveValuesAllowed": [],
  "servicesNames": [
    "nifi-service",
    "argo-server",
    "minio",
    "postgres",
    "workflow-controller-metrics",
    "weave-scope-app",
    "kubernetes-dashboard"
  ],
  "untrustedRegistries": [],
  "wlKnownNames": [
    "coredns",
    "kube-proxy",
    "event-exporter-gke",
    "kube-dns",
    "17-default-backend",
    "metrics-server",
    "ca-audit",
    "ca-dashboard-aggregator",
    "ca-notification-server",
    "ca-ocimage",
    "ca-oracle",
    "ca-posture",
    "ca-rbac",
    "ca-vuln-scan",
    "ca-webhook",
    "ca-websocket",
    "clair-clair"
  ]
}

I've also tried adding the exceptions by their name but I got the same result. And also adding the execeptions as a list like so:
{
        "name": "my-control-exceptions-list",
        "policyType": "postureExceptionPolicy",
        "actions": [
            "alertOnly"
        ],
        "resources": [
            {
                "designatorType": "Attributes",
                "attributes": {
                    "kind": ".*"
                }
            }
        ],
        "posturePolicies": [
            {
                "controlID": "C-0004"
            },
            {
                "controlID": "C-0012"
            },
            {
                "controlID": "C-0055"
            },
            {
                "controlID": "C-0076"
            },
            {
                "controlID": "C-0193"
            },
            {
                "controlID": "C-0202"
            },
            {
                "controlID": "C-0211"
            },
            {
                "controlID": "C-0225"
            },
            {
                "controlID": "C-0236"
            },
            {
                "controlID": "C-0237"
            }
        ]
    }

Even adding my exceptions to the exceptions.json file downloaded with the artifacts (when using local ones) returns the same result.
Thank you!

Thank you @inakizorrilla for reporting, I will try to reproduce this.
Can you please share the repository you tried testing? (if it is public...)

I am afraid I cannot share the repo, but I can paste the obfuscated output of the scanned kustomize build.

Kustomize build file
apiVersion: v1
kind: Service
metadata:
  name: myappname
  namespace: ${KUBERNETES_NAMESPACE}
spec:
  ports:
  - port: 8080
  selector:
      app: myappname
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: myappname
  namespace: ${KUBERNETES_NAMESPACE}
spec:
  minReadySeconds: 5
  replicas: 1
  selector:
      matchLabels:
          app: myappname
  strategy:
      rollingUpdate:
          maxSurge: 2
          maxUnavailable: 0
  template:
      metadata:
          labels:
              aadpodidbinding: ${POD_IDENTITY_ID}
              app: myappname
      spec:
          containers:
          - env:
              - name: APP_ENVIRONMENT
                value: Development
           image: ${CONTAINER_REGISTRY_NAME}.azurecr.io/myappname
           imagePullPolicy: Always
           name: myappname
           ports:
           - containerPort: 8080
           readinessProbe:
               failureThreshold: 3
               httpGet:
                   path: /health
                   port: 8080
                   scheme: HTTP
               initialDelaySeconds: 5
               periodSeconds: 5
               successThreshold: 1
               timeoutSeconds: 1
           resources:
               limits:
                   cpu: 300m
                   ephemeral-storage: 500Mi
                   memory: 500Mi
               requests:
                   cpu: 50m
                   ephemeral-storage: 100Mi
                   memory: 100Mi
           securityContext:
               allowPrivilegeEscalation: false
               capabilities:
                   drop:
                   - all
               readOnlyRootFilesystem: true
               runAsGroup: 30000
               runAsUser: 10000
           volumeMounts:
           - mountPath: /tmp
             name: tmp
           - mountPath: /mnt/secrets-store###1
             name: secrets-store###1
             readOnly: true
     volumes:
     - emptyDir: {}
       name: tmp
     - csi:
           driver: secrets-store.csi.k8s.io
           readOnly: true
           volumeAttributes:
               secretProviderClass: azure-kv-secretprovider###1
        name: secrets-store###1

If you need anything else please ask me.
Thank you for your time!

I am able to reproduce similar issue for the controls C-0211
> kubescape scan control C-0211  . --exceptions kubescape.json
[info] Kubescape scanner starting
[...]

Controls: 1 (Failed: 1, Passed: 0, Action Required: 0)
Failed Resources by Severity: Critical — 0, High — 7, Medium — 0, Low — 0

+----------+----------------------------------------------------+------------------+---------------+--------------------+
| SEVERITY |                    CONTROL NAME                    | FAILED RESOURCES | ALL RESOURCES | % COMPLIANCE-SCORE |
+----------+----------------------------------------------------+------------------+---------------+--------------------+
| High     | Apply Security Context to Your Pods and Containers |        7         |       7       |         0%         |
+----------+----------------------------------------------------+------------------+---------------+--------------------+
|          |                  RESOURCE SUMMARY                  |        7         |       7       |       0.00%        |
+----------+----------------------------------------------------+------------------+---------------+--------------------+

> kubescape version
Your current version is: v2.3.2 [git enabled in build: true]

Exceptions file:
[
    {
        "name": "exclude-resources-from-controls",
        "policyType": "postureExceptionPolicy",
        "actions": [
            "alertOnly"
        ],
        "attributes": {
            "systemException": true
        },
        "resources": [
            {
                "designatorType": "Attributes",
                "attributes": {
                    "name": ".*"
                }
            }
        ],
        "posturePolicies": [
            {
                "controlID": "C-0211"
            }
        ]
    }
]

Hi @inakizorrilla @rtalipov! Has this issue been resolved or does it still persist?

Hi,
I have been unable to further test this because without this feature the tool wasn't useful to me.

Just tested this out, this still persists, I'll take a look into this.

@dwertent @fredbi apparently the issue was because of the opa-utils repo instead of this one.
I've opened a PR linked above in that repo that fixes this bug ;)


Unexpected response body when using WebClient behind a proxy server

I'm at a bit of a loss as to what the issue is here as my code as been working for some time.  We are leveraging the WebClient behind a corporate proxy and have set the proxy parameter in the call.  Starting today we were experiencing SSL issues which we appear to have resolved.  However, now we are running into a JSONDecoder Issue and am at a loss for what the issue may be.
When manually running a curl command we have no issues
curl -X POST https://slack.com/api/api.test\?token\='xoxb....

However when running the WebClient we are experiencing the following issue (included the logging DEBUG for this step as well):
SLACK_CLIENT = slack.WebClient(token='xoxb...',proxy='<corporate proxy>:8099',base_url='https://slack.com/api/')

DEBUG:slack.web.base_client:Sending a request - url: https://slack.com/api/auth.test, query_params: {}, body_params: {}, files: {}, json_body: {}, headers: {'Content-Type': 'application/json;charset=utf-8', 'Authorization': '(redacted)', 'User-Agent': 'Python/3.8.0 slackclient/2.9.0 Linux/4.14.186-146.268.amzn2.x86_64'}
Traceback (most recent call last):
  File "/home/git/.local/lib/python3.8/site-packages/slack/web/base_client.py", line 355, in _urllib_api_call
    response_body_data: dict = json.loads(response["body"])
  File "/opt/runtime/python-3.8/lib/python3.8/json/__init__.py", line 357, in loads
    return _default_decoder.decode(s)
  File "/opt/runtime/python-3.8/lib/python3.8/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/opt/runtime/python-3.8/lib/python3.8/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

Any assistance would be greatly appreciated

This is just my guess but probably the response body might be empty for some reason. What happens if you go with other HTTP client library (e.g., requests)?
>>> s = ""
>>> import json
>>> json.loads(s)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/ksera/.pyenv/versions/3.8.5/lib/python3.8/json/__init__.py", line 357, in loads
    return _default_decoder.decode(s)
  File "/Users/ksera/.pyenv/versions/3.8.5/lib/python3.8/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/Users/ksera/.pyenv/versions/3.8.5/lib/python3.8/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

@seratch Thanks for the quick response.  Using the requests library returns what I would expect using the same values triggered in the same way
url = 'https://slack.com/api/auth.test'
headers = {'Content-Type':'application/json;charset=utf-8', 'Authorization': 'Bearer xoxb....'}
r= requests.post(url, headers=headers)
print(r.text)

DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): slack.com:443
DEBUG:urllib3.connectionpool:https://slack.com:443 "POST /api/auth.test HTTP/1.1" 200 166
{"ok":true,"url":"<url>","team":"<Team>","user":"<user>","team_id":"<id>","user_id":"<id>","bot_id":"<id>","enterprise_id":"<id>"}

@jmgreg31 Thanks for your prompt response too! So, you're using env variables (HTTPS_PROXY, HTTP_PROXY) along with requests library, right?
Although I'm still not sure yet, I am thinking about one possibility. If the proxy requires authentication like https://github.com/slackapi/python-slackclient/issues/743 and the URL format is like http://user:pass@example.com:12345, you might be affected by the unsolved issue. The behavior looks a bit different in your case but that may depend on the proxy server's behavior. If this is your situation, a possible workaround at this point is to use WebClient(run_async=True or AsyncWebClient.

@seratch I double checked that the proxies were set for the requests call as well.
However, using your recommendation of run_async=True seemed to resolve the issue.
SLACK_CLIENT = slack.WebClient(token='<token>',
                               proxy='<proxy>',
                               base_url='https://slack.com/api/',
                               run_async=True)
TEST_SLACK_CLIENT = SLACK_CLIENT.auth_test()

loop = asyncio.get_event_loop()
try:
    response = loop.run_until_complete(TEST_SLACK_CLIENT)
    print(response)
except SlackApiError as e:
    print(f"Got an error: {e.response['error']}")
finally:
    loop.close()

Really appreciate the recommendation!

@jmgreg31 Sorry for this inconvenicence. Using HTTPS_PROXY env variable can be a much better workaround. See also: https://github.com/slackapi/python-slackclient/issues/820#issuecomment-697156651

This issue has been addressed in version 2.9.1. We're sorry about this. https://github.com/slackapi/python-slackclient/releases/tag/v2.9.1


Incorrect return type from createSelector when selector returns union of types with keys in common and undefined

First off, thanks for the reduxjs project!
Here is a description of what I think is a bug.
See this repro for a reproduction. Reproduced with<EMAIL_ADDRESS>NOK
When a selector returns a type A | B | undefined, and types A and B have a key in common, the return type of the selector is A | undefined instead of A | B or undefined.
OK
When types A and B do not have a key in common, the return type is correct, i.e. A | B | undefined.
OK
When types A and B have a key in common, but the return union type does not include undefined, the return type is correct, i.e. A | B.

this is normal TS behaviour - B is a subtype of A (any B is assignable to A), so TS simplifies to the widest type.

Thanks for the swift response. You're quite right. I hadn't even considered it might be coming from TS and that it is normal behaviour.


malhar-angular-dashboard does not support latest version of angular-ui-bootstrap

$modal is deprecated since angular-bootstrap#0.14.0
I recommend to use $uibModal instead.

Edit*:
See https://github.com/DataTorrent/malhar-angular-dashboard/pull/161
All checks have passed. A merge would be nice :)

This seems resolved by:
https://github.com/DataTorrent/malhar-angular-dashboard/pull/164
Can this issue be closed?

Yes it can.
Thanks for merging my pull request.


python_testing: Use Command ID checker in TC_DeviceBasicComposition

This PR uses the Command ID checker to improve the error message on command id related failures.
It adds "(Test Vendor)" to the error message in case a test vendor id is used.
This is common in our example apps.

Waiting for https://github.com/project-chip/connectedhomeip/pull/36024 and https://github.com/project-chip/connectedhomeip/pull/36018 to land so I can merge master and send this out for review.


Bug: No matching export in "vite-plugin-node-polyfills/shims/process/dist/index.js" for import "env"

Summary
I'm facing this error:
✘ [ERROR] No matching export in "node_modules/vite-plugin-node-polyfills/shims/process/dist/index.js" for import "env"

    node_modules/.pnpm/is-in-ci@1.0.0/node_modules/is-in-ci/index.js:1:8:
      1 │ import {env} from 'node:process';
        ╵         ~~~

✘ [ERROR] No matching export in "node_modules/vite-plugin-node-polyfills/shims/process/dist/index.js" for import "cwd"

    node_modules/.pnpm/ink@5.1.0_@types+react@18.3.17_react-devtools-core@6.0.1_react@18.3.1/node_modules/ink/build/components/ErrorOverview.js:2:9:
      2 │ import { cwd } from 'node:process';

I can fix this by adding:
export const cwd = process.cwd;

For now, I've solved this by setting this in my vite config:
    config.plugins.push(
      nodePolyfills({
        globals: {
          process: false,
        },
        overrides: {
          process: require.resolve('../mocks/node-process.ts'),
          module: require.resolve('../mocks/node-module.ts'),
          util: require.resolve('../mocks/node-util.ts'),
          stream: require.resolve('stream-browserify'),
          buffer: require.resolve('buffer/'),
          assert: require.resolve('assert/'),
          os: require.resolve('os-browserify/browser'),
          tty: require.resolve('tty-browserify'),
        },
      }),
      topLevelAwait()
    );

And this is my mock:
export * from 'vite-plugin-node-polyfills/shims/process';

import process from 'vite-plugin-node-polyfills/shims/process';
export default process;

export const cwd = () => '/';
export const env = {};

I had to disable globals.process in order for it to accept the overrides.process.
FYI, I'm doing something rather niche/experimental, so don't worry if this doesn't get fixed.


Example Code

Hello,
i noticed that the example is not working for me, it register a require('.') but need require('hapi-pino'), i suppose that in the package it refer to itself so there is fine.
Is my assumption correct?
Thanks.

that's right - the example is intended to be run in the repo itself
this is a commonly occurring approach to example.js files

this is stupid documentation

@geforcesong, I've censored your comment. Please be polite.


Update README with examples and FAQ

Removed some old and slightly confusing examples, and instead focus on the basic CLI usage.

@ww-daniel-mora Hey Daniel, I got you something.

Thanks @philsturgeon

@ww-daniel-mora fancy some beta access for v3<EMAIL_ADDRESS>and I'll sort it out.


[SPARK-8918] [MLLIB] [DOC] Add @since tags to mllib.clustering

This continues the work from #8256. I removed @since tags from private/protected/local methods/variables (see https://github.com/apache/spark/commit/72fdeb64630470f6f46cf3eed8ffbfe83a7c4659). @MechCoder
Closes #8256

Merged build triggered.

Merged build started.

Test build #41166 has started for   PR 8288 at commit 72fdeb6.

Test build #41166 has finished for   PR 8288 at commit 72fdeb6.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
   implicit class StringToColumn(val sc: StringContext)

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41166/
Test PASSed.

I made a pass and sent a Pull Request adding the missing tags. Please review and merge the changes (https://github.com/mengxr/spark/pull/3)

@yu-iskw It would be great if you could hold on your comments till @mengxr merges my pull request with this branch :) After that you could make a final pass

@MechCoder Alright. Thank you for letting me know.

Merged build triggered.

Merged build started.

@mengxr thanks for the merge. @yu-iskw please give a final pass now. (I added quite a few missing tags)

Merged build triggered.

Merged build started.

Test build #41247 has started for   PR 8288 at commit d2e4849.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41246/
Test FAILed.

Test build #41247 has finished for   PR 8288 at commit d2e4849.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41247/
Test PASSed.

LGTM, minus above two things.

Merged build triggered.

Merged build started.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41254/
Test FAILed.

test this please

Merged build triggered.

Merged build started.

Test build #41268 has started for   PR 8288 at commit b5b9168.

Test build #41268 has finished for   PR 8288 at commit b5b9168.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41268/
Test PASSed.

Merged into master and branch-1.5. Thanks @XiaoqingWang for the first version and @yu-iskw @MechCoder for review!
@XiaoqingWang Could you ping me on the JIRA page? So I can assign the ticket to you.


What happens if Security Group is deleted in Office Admin site?

The Security Group cannot currently be de-associated, but what if the actual Group is deleted in office admin?
What would be the expected behavior? De-associate?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 9aafd67e-8013-565e-6566-9773136c4405
Version Independent ID: 1db48adc-6512-7978-4656-2cf081b5cdb7
Content: Control user access to environments: security groups and licenses - Power Platform Admin center
Content Source: power-platform/admin/control-user-access.md
Service: power-platform
GitHub Login: @jimholtz
Microsoft Alias: jimholtz

@AgileXRM
Thanks for bringing this to our attention. We're aware of this issue and working on a fix.


File picker browse option

Hi Martijn,
We recently pulled the latest enketo code and observed that browse option is removed for media type questions. I think it is not much intuitive if we do not have browse option . But I want to know what is your though process around it.
Also I observed for date question when appearance is month-year, in enketo form we show YYYY-MM instead of MM-YYYY. Is there any reason behind the same.
Thanks,
Deepthi Chand. A

Hi Deepthi,
You mean the little button that you normally find in native HTML file inputs, right? This was changed to look like a regular input (basically a fake file input) to allow for easier Cordova-wrapping where the file inputs are overridden using a native hook.  However, we could perhaps put a (fake) button back if that makes the UI better. I quite like that they look the same as number and text inputs, but am not sure what is best for users.
If I remember correctly, the date input formats were chosen to:

stay close to the ISO format used in the datatype
have a format that will be least confusing between different locales (e.g. in the USA 01-07-2015 is the 1st of July not the 7th of January).

Ideally we should localize these input formats, but that's not something I personally look forward to doing :P.

Ah, I realize you might be referring to the name of the appearance! Yes, it would be more consistent to use appearance="year-month". That was an early choice in the XForm format development. We should probably leave that now.

downloads disabled in IE11 forever
downloads disabled in MS Edge until browser bug is fixed

bcaf2e477185eaf6fa90a11f0796420f399a757f

[ ] Edge workaround?
[ ] max size async workaround?
[ ] loading file from storage not showing download button

https://stackoverflow.com/questions/24007073/open-links-made-by-createobjecturl-in-ie11

the dataURI download seems to also fail for a 3.2 MB file


Update to support nan 2.*

Building will still fail until v8.h (chakra) has FunctionTemplate methods SetCallHander and SetCallAsFunctionHandler.

cc @jianchun

LGTM.
@kunalspathak Kunal, could you please CR this and follow up?

LGTM. Yes, I will address the issue in node-msft


fix: allow CardContent to pass through style to Tile component

Description
In some instances it may be necessary to pass a style down from CardContentHorizantal through the tile prop into the Tile component. This
References
No Jira
Testing
Open CardContentHorizantal story
remove all controls to ensure you can add the required props directly to the component. Then add something like the following.
        tile: {
            style: {
              paddingX: 40
            },
            progressBar: {
              progress: .5
            },
            metadata: {
              title: 'foo'
            }
          }

All tile styles should now be allowed to be modified at the instance level.
Automation

Checklist

[ ] all commented code has been removed
[ ] any new console issues have been resolved
[ ] code linter and formatter has been run
[ ] test coverage meets repo requirements
[ ] PR name matches the expected semantic-commit syntax

Test Execution Passed.

Test Execution Failed.

Test Execution Failed.

:tada: This PR is included in version @lightningjs/ui-components-v2.20.5 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


[libra-dev] Adding core client library crate

Motivation
This PR takes the first step in adding a core client library by creating a libra-dev crate with an initial deserialize account resource class
Test Plan
tbd

Closing since we're moving to a new repo


Amazing blazing pizza's

Thank you for the great workshop guys,
Keep it up !

Thank you for the great workshop guys,
Keep it up !

@DimitriVanhaecke ,
Since this is not an issue, can you close this?

Thank you for the great workshop guys,
Keep it up !

@DimitriVanhaecke ,
Since this is not an issue, can you close this?


Inconsistent output in jsdom and browser.

var jsdom = require('jsdom');

var DISNEY_PRINCES_URL = 'http://princess.disney.co.uk/';

jsdom.env({
  url: DISNEY_PRINCES_URL,
  done: function (errors, window) {
    var document = window.document;
    var entityLinks = document.querySelectorAll('.slides .entity-link.cta-link:not([href*=video])');
    [].slice.call(entityLinks).forEach(function(entityLink){
      console.log(entityLink.getAttribute('href'));
    });
  }
});

logs:
http://princess.disney.co.uk/cinderella
http://princess.disney.co.uk/tiana
http://princess.disney.co.uk/ariel
http://princess.disney.co.uk/aurora

However, in browser
var entityLinks = document.querySelectorAll('.slides .entity-link.cta-link:not([href*=video])');
[].slice.call(entityLinks).forEach(function(entityLink){
  console.log(entityLink.getAttribute('href'));
});

logs:
http://princess.disney.co.uk/cinderella
http://princess.disney.co.uk/tiana
http://princess.disney.co.uk/ariel
http://princess.disney.co.uk/aurora
http://princess.disney.co.uk/belle
http://princess.disney.co.uk/rapunzel
http://princess.disney.co.uk/jasmine
http://princess.disney.co.uk/merida
http://princess.disney.co.uk/mulan
http://princess.disney.co.uk/snow-white
http://princess.disney.co.uk/pocahontas

That's because Disney doesn't load elements outside of the screen space into the list. You can simulate this by resizing your browser window to a small area, reloading the page and executing the script again.
You'll need to "load"/navigate to the next page of princesses.

Thank you @Sebmaster.


Loan application feature

I added the loan application feature

This pull request is trying to merge into master branch instead of develop branch


Add Rpi.GPIO to pip dependencies

In some systems like mine, the status-led.service required me to install this pip package outside the virtualenv.

One of the steps I had to take to fix #23


Microsoft.Azure.Management.Storage 7.0.0

Description

This checklist is used to make sure that common guidelines for a pull request are followed.

[x] I have read the contribution guidelines.
[ ] The pull request does not introduce breaking changes.

General Guidelines

[x] Title of the pull request is clear and informative.
[x] There are a small number of commits, each of which have an informative message. This means that previously merged commits do not appear in the history of the PR. For more information on cleaning up the commits in your PR, see this page.

Testing Guidelines

[x] Pull request includes test coverage for the included changes.

SDK Generation Guidelines

[x] If an SDK is being regenerated based on a new swagger spec, a link to the pull request containing these swagger spec changes has been included above.
[x] The generate.cmd file for the SDK has been updated with the version of AutoRest, as well as the commitid of your swagger spec or link to the swagger spec, used to generate the code.
[x] The *.csproj and AssemblyInfo.cs files have been updated with the new version of the SDK.

https://github.com/Azure/azure-rest-api-specs/pull/1494

https://github.com/Azure/azure-rest-api-specs/pull/1579

hi @shahabhijeet , Spec PR link and description are added.
There is a breaking change in this PR(renamed networkAcl to networkRuleSet).
Though this is only a preview SDK, we usually bump up major version once there is breaking change.


Making littlefs thread-safe?

Hey! I just found littlefs and am porting it this morning to a FreeRTOS system with 128MB NOR flash.
I see that littlefs doesn't have any sort of thread-safe mechanism built in. That's fine and I've seen references to wrapping littlefs calls with semaphores. Also fine.
The question is which calls need to be wrapped. open/close/read/write/??
Anything that has a prototype in lfs.h? That'll likely be my first stab at it, but perhaps this is an issue that someone else has resolved in an elegant manner..
Thanks for any insights.

The question is which calls need to be wrapped. open/close/read/write/??

Everything which is an user-facing public API.
For an example see this port:
https://github.com/DISTORTEC/distortos/tree/master/source/FileSystem/littlefs
https://github.com/DISTORTEC/distortos/tree/master/include/distortos/FileSystem/littlefs

Unfortunately I have zero experience with C++. Is "const std::lock_guard lockGuard {*this};" a semaphore "take"?
Is this "return" the semaphore "give" "return {{}, ret};"

Is "const std::lock_guard lockGuard {*this};" a semaphore "take"?

Yes.

Is this "return" the semaphore "give" "return {{}, ret};"

Not exactly. Any possible exit from this function will destroy this lockGuard object, which will cause its destructor to be executed, which in turn releases the mutex. Generally any exit from this function causes automatic release of the mutex.
Please also don't confuse mutex and semaphore - the fact that in FreeRTOS they are "almost the same thing" (one of many stupid things in FreeRTOS...) doesn't mean that they are meant for the same purpose.

Thanks.
It appears that there is a different mutex used for directories than for files. Do you perhaps know if this is true (again, I don't really grok C++) and if so, why?

There are no separate mutexes for file and directory objects - there is only one recursive mutex per file system and locking the file or directory is achieved by locking the mutex in the associated file system.
lockGuard is only a RAII (look it up) object which locks/unlocks the mutex/object that you provide. The "thing" you provide just has to have lock() and unlock() functions which will be called in constructor and destructor respectively.
Here's the only mutex in these clases:
https://github.com/DISTORTEC/distortos/blob/master/include/distortos/FileSystem/littlefs/LittlefsFileSystem.hpp#L290
Note that there is no such mutex in file or directory classes. If you lock the file manually this calls lock() of associated file system:
https://github.com/DISTORTEC/distortos/blob/master/source/FileSystem/littlefs/LittlefsFile.cpp#L100
which in turn just locks the common mutex:
https://github.com/DISTORTEC/distortos/blob/master/source/FileSystem/littlefs/LittlefsFileSystem.cpp#L307
But you don't have to manually lock anything. Each function does its own locking (with the std::lock_guard). The lock() and unlock() functions are required only if you want to perform several operations "atomically", for example you need to do 10 writes back-to-back and don't want any write from other thread to mix things up (for example you are writing to a shared log file but you cannot write the complete entry with one call and need several of them).

Thanks. C++ continues to seem unnecessarily obscure and unnecessarily complex to me. I suppose that comes from learning to program back when "C" was a new language. :)
Can I distill down that what I need to do for each of the lfs externally visible functions to something like this:
int my_getsize(
{
xSemaphoreTake( fs_mutex,0xffffffff);
int size = lfs_file_size(&fileSystem_.fileSystem_, &file_);
xSemaphoreGive( fs_mutex );
return size;
}

Yes.

Thanks for the help.

Sounds like everything got sorted out here? Thanks @FreddieChopin! Let me know if this issue needs to be reopened.

Yes - everything is working fine. As a future feature request it would be a lot easier to do thread-safe if there was the possibility of a "get a mutex" and "release a mutex" function configuration, and LittleFS called them itself at the appropriate places.

Not a bad idea, assuming 1. out of the box had default stubs (similar to lfs_malloc) and 2. stubs did not increase existing code size.
Unfortunately, this is low priority at the moment. I'd be happy to take in a PR, but I think there's some bigger issues to take care of first.

This is just an update of @FreddieChopin 's links:
Mutex:
https://github.com/DISTORTEC/distortos/blob/79062407feef8b80b28929610f4eb4e13e467b29/include/distortos/FileSystem/Littlefs1FileSystem.hpp#L309
File/Directory's lock():
https://github.com/DISTORTEC/distortos/blob/746afae4c4d8485c9aafc572364861c6ac6de223/source/FileSystem/littlefs1/Littlefs1File.cpp#L94-L97
Filesystem-wide lock():
https://github.com/DISTORTEC/distortos/blob/746afae4c4d8485c9aafc572364861c6ac6de223/source/FileSystem/littlefs1/Littlefs1FileSystem.cpp#L265-L269

Hello  @rbphilip, could you provide us an example of the wrapper you did?  I have the same situation here and I have not clue how to do that thread-safe wrapper.
Thank you

Sorry. That was several clients ago.
But basically create wrapper functions for all the littlefs calls, and
use a semaphore to "lock" access to the littlefs library.. something
like below.
somewritefunction (data, size)
{
     take semaphore
     call littlefs function to write
     release semaphore
}
On 2021.09.17 11:43, johnnytolengo wrote:

Hello @rbphilip https://github.com/rbphilip, could you provide us
an example of the wrapper you did? I have the same situation here
and I have not clue how to do that thread-safe wrapper.
Thank you
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/littlefs-project/littlefs/issues/156#issuecomment-921971364,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABA7NPV2HD2XV3TSAG3PHBLUCN45HANCNFSM4G6QQ6MA.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

yes, but that is already build in look here: https://github.com/lorol/LITTLEFS/blob/master/src/esp_littlefs.c#L662
I think the only problem is around the file descriptor when the file is opened multiple times: https://github.com/joltwallet/esp_littlefs/pull/19
I am not sure if that is already solved in the https://github.com/lorol/LITTLEFS
I it's all about thread-safty  :(

It's been years since I even thought of littlefs.
On 2021.09.17 12:38, johnnytolengo wrote:

yes, but that is already build in look here:
https://github.com/lorol/LITTLEFS/blob/master/src/esp_littlefs.c#L662
https://github.com/lorol/LITTLEFS/blob/master/src/esp_littlefs.c#L662
I think the only problem is around the file descriptor when the file
is opened multiple times: joltwallet/esp_littlefs#19
https://github.com/joltwallet/esp_littlefs/pull/19
I am not sure if that is already solved in the
https://github.com/lorol/LITTLEFS https://github.com/lorol/LITTLEFS
I it's all about thread-safty :(
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/littlefs-project/littlefs/issues/156#issuecomment-922002878,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABA7NPQJPWCK76DJNMZWM7LUCODKVANCNFSM4G6QQ6MA.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

@rbphilip :

It's been years since I even thought of littlefs.

Have you maybe found some nice alternatives to it? (;

Sorry, no.
It worked well for the client at the time but I haven't had need for a
file system on any projects since then.
On 2021.09.17 15:41, Freddie Chopin wrote:

@rbphilip https://github.com/rbphilip :
It's been years since I even thought of littlefs.

Have you maybe found some nice alternatives to it? (;
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/littlefs-project/littlefs/issues/156#issuecomment-922097053,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABA7NPQY3KXO3V7XKDCMJJDUCOYX7ANCNFSM4G6QQ6MA.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.


Issue by following the dotSpatial tutorial 4 in "Create a Point shapefile" part.

Hello,
I want to create a point on my shapefile so i followed the tutorial on DotSpatial website but i have an issue on this part:
switch (shapeType)             {                 case  "Point" :                     if (e.Button == MouseButtons.Left)                     {                         if ((pointmouseClick))                         {                             Coordinate coord = map1.PixelToProj(e.Location);                              DotSpatial.Topology.Point point = new DotSpatial.Topology.Point(coord);                             IFeature currentFeature = pointF.AddFeature(point);                             pointID = pointID + 1;                             currentFeature.DataRow["PointID"] = pointID;                              map1.ResetBuffer();                         }                     }
I have an error on these lines, which says "You need to add the assembly DotSpatial.GeoApi"
 Coordinate coord = map1.PixelToProj(e.Location);
and on this line too
 IFeature currentFeature = pointF.AddFeature(point);
Further more if i use the assembly DotSpatial.GeoApi visual studio says "Unable to convert GeoAPI.Geometries.Coordinate to DotSpatial.Topology.Coordinate"
I don't find where i did it wrong, if someone can help me.
Faithfully
Clément

When you declare your Coordinate variables make sure you use the full namespace.

@Tisscl The example you used was based on DS 1.9 or earlier. DS 2.0 no longer contains Dotspatial.Topology.
If you're using the dlls from DS 1.9 (e.g from nuget) or earlier you'll need to reference Dotspatial.Topology.
If you're using dlls from DS 2.0 (from github) you'll need to replace every mention of Dotspatial.Topology from the example by either GeoAPI or NetTopologySuite depending on where the things from Topology were moved to.
Point for example was moved to NetTopologySuite.Geometries.

Ok thank you for your answers !
I have no more errors on these lines but i have one error on this line :
NetTopologySuite.Geometries.Point point = new point(coord)
It say "Unable to convert GeoAPI.Geometries.Coordinate to System.Drawing.Size"
Did i mess something else ?

it should be
NetTopologySuite.Geometries.Point point = new NetTopologySuite.Geometries.Point(coord)

Oh my bad, thank you for your help it works now !


HDRP: double sided and alpha cutout are not set correctly for this model

Example model:
https://sketchfab.com/3d-models/grass-variations-c84573687bf14f89938002df4ca0e696#download
After initial import:

Now enable double-sided and alpha clipping in HDRP Lit, render looks correct:

And to be clear, doubleSided is set on the USD mesh:


Job Bot only answers on new needs

The job bot does not seem to answer on needs it created before a restart.

Does the restart also resets the mongoDB entries for the bot?

closed in favor of specific item within the jobbot repo)


Fixed package license identifier.

I believe the LICENSE file content to be correctly identified by BSD-3-Clause SPDX identifier.

ping @ericf

I've emailed Eric in case he has missed the notification as I often do.

@ericf any chance you can do a release with this included?

@ericf are you able to do a release with this included?


faa-passing-reverse animation

I couldn't find a easy way to rotate this animation 180 degrees so I simply added faa-passing-reverse to change the direction.

:+1:


FFTView: Create View Without Normalization

Set the FFTTap.isNormalized to false to stop the normalization of the fftData. Tweaked the mapping of the returned fftData to match the visualizer.


Documentation is broken

The new API docs can not be used correctly.
A click on a item in left side navigation menu does refresh the whole page and shows the item link, before reload you can see that the submenu was opened, too.
=> Fix it that click on item opens submenu, because currently it is not possible to get to a submenu point directly.
Also the search seems to be broken, too. I can't search for "Fires" or can't find an overview about events.
And maybe there are some methods missing: Search for "addLayer" and you only get the PluggableMap Method "addLayer", but a addLayer on a normal map instance still works?!!!
So there are a lot of small issues, or did I miss something

Thanks for the report! We are in the process of reworking our JSDoc annotations (#8345). When done, we will also switch to a different doc generator. So these issues should be fixed for the next release.

Thanks for Quick response. I will wait for next release now!


Remove version: Bitvise.SSH.Client version 9.33

:warning: Vanity URL
This package uses a vanity URL, which always points to the latest version. Since a newer version is released, outdated versions should be removed from the repository.

Resolves #198832

Microsoft Reviewers: Open in CodeFlow

URL: https://dl.bitvise.com/BvSshClient-Inst.exe
OK
(Automated message - build 923)


[question] Why Ranger is not available as a pip package

Why Ranger is only available on a github but not as an pip package. Wouldn't it be easier for the community to actually use it?

Actually, there is a pip package but it is based out of a fork of this repo. I think it would make sense to collate this effort to the main repo.

I'm currently 3/4 done with a rewrite of Ranger (Ranger21) to include a number of new innovations since Ranger was released.  I will make that a pip package once done and also include the original Ranger as well to address the lack of easy install.
https://github.com/lessw2020/Ranger21 is the repo for Ranger21 if you want to take a look at what's coming.

Awesome! I can contribute the conda recipe, if you want.


Lsp init fix

Some background:
https://github.com/coc-extensions/coc-omnisharp/issues/33
https://github.com/coc-extensions/coc-omnisharp/issues/35
https://github.com/OmniSharp/omnisharp-roslyn/issues/1742
https://github.com/OmniSharp/omnisharp-roslyn/issues/1515
https://github.com/OmniSharp/omnisharp-roslyn/issues/1083

In short, there's race condition between (workspace initialization + project reference adding) and (client requests coming in at a random point of time).
This PR blocks incoming LSP requests until the initial project system processing is completed. Should help us to get rid of a bunch of issues.

@yatli Sorry for the delay.
I got looking into this.
I think I'm okay with the addition to the project system @filipw or @JoeRobich if you could comment if you have any issues with the functionality (or 🚲 🏠 ).
The plumbing through the initialized handler looks like a code smell to me.
There is a better solution but because my documentation on the csharp-lsp library is limited, I'm sure you just didn't know about it.
I've made a single commit to this branch: https://github.com/OmniSharp/omnisharp-roslyn/tree/dd-lsp-init-proposed-fix with the changes to use the OnInitialized even to wait for the workspaces to be started.  This event fires just before the server returns the initialize response to the client.

If you can give it a test spin for me to make sure it solves your issue, then we can go with that.

Thanks @david-driscoll !
Will test it out and report back.

Can confirm it solves the initial loading problem.

@david-driscoll shall I merge your branch?

Thanks @yatli !


Add NusaParagraph Emotion Classification

NusaParagraphEmotionClassification is a multi-class emotion classification on 10 Indonesian languages.
Checklist

[x] Run tests locally to make sure nothing is broken using make test.
[x] Run the formatter to format the code using make lint.

Adding datasets checklist

Reason for dataset addition: ...

[x] I have run the following models on the task (adding the results to the pr). These can be run using the mteb -m {model_name} -t {task_name} command.

[x] sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2
[x] intfloat/multilingual-e5-small

[x] I have checked that the performance is neither trivial (both models gain close to perfect scores) nor random (both models gain close to random scores).
[x] If the dataset is too big (e.g. >2048 examples), considering using self.stratified_subsampling() under dataset_transform()
[x] I have filled out the metadata object in the dataset file (find documentation on it here).
[x] Run tests locally to make sure nothing is broken using make test.
[x] Run the formatter to format the code using make lint.

cc @Muennighoff @KennethEnevoldsen

Thank you for the review. I addressed them. I updated the avg number of character per sample, added the results, and points. I calculated the points, so there are new languages, but it only covered by other PR https://github.com/embeddings-benchmark/mteb/pull/927. So the points are 2. Plus, I added the point for the reviewer. @KennethEnevoldsen


port to love 11.3

Hi, I don't know if you're interested, but I made some little patches to make the game compatible with love 11.3.
By the way it's a very nice game and very useful for learning.
Bye
Michele

Hello Michele, many thanks for contributing this update, and sorry i didn't get back to you sooner!
I've checked and merged in your PR for 11.3. I note that 11.4 is also out now, but this PR probably gets the code most of the way there.
Thanks again!

Hi, I'm glad it was useful!
Bye
Michele
Il 25/01/23 16:22, Krunkathos ha scritto:

Hello Michele, many thanks for contributing this update, and sorry i
didn't get back to you sooner!
I've checked and merged in your PR for 11.3. I note that 11.4 is also
out now, but this PR probably gets the code most of the way there.
Thanks again!
—
Reply to this email directly, view it on GitHub
https://github.com/krunkathos/quasimofo/pull/1#issuecomment-1403788968, or unsubscribe https://github.com/notifications/unsubscribe-auth/AC35VXSLFOGSMPRM7OGQ4ZLWUFAMZANCNFSM5PBB2A5Q.
You are receiving this because you authored the thread.Message ID:
@.***>


Minimise rate limiting

Investigate whether we can:

reduce API calls
set concurrency limits
other

We can investigate the graphQL API to collect more data in one call.
Something like:
  const query = `
    query($owner: String!, $name: String!) {
      repository(owner: $owner, name: $name) {
        createdAt
        pushedAt
        object(expression: "HEAD:package.json") {
          ... on Blob {
            text
          }
        }
        packageLock: object(expression: "HEAD:package-lock.json") {
          ... on Blob {
            text
          }
        }
        yarnLock: object(expression: "HEAD:yarn.lock") {
          ... on Blob {
            text
          }
        }
      }
    }
  `

would get the repo metadata, and the package file in one go. (I don't know GraphQL...)


chore(deps): update dependency remark-cli to v6

This PR contains the following updates:

Package
Type
Update
Change
References

remark-cli
devDependencies
major
^5.0.0 -> ^6.0.0
homepage, source

Release Notes

remarkjs/remark
v6.0.0
Compare Source
Better virtual files! Better CLI!
Virtual files
Read more about that in<EMAIL_ADDRESS>Configuration Presets on the CLI
Presets are now supported on the CLI. They‘re basically configuration files published to npm.
Read more about those in<EMAIL_ADDRESS>

Renovate configuration
:date: Schedule: At any time (no schedule defined).
:vertical_traffic_light: Automerge: Disabled by config. Please merge this manually once you are satisfied.
:recycle: Rebasing: Whenever PR is stale, or if you modify the PR title to begin with "rebase!".
:no_bell: Ignore: Close this PR and you won't be reminded about this update again.

This PR has been generated by Renovate Bot.

PR has been edited
:construction_worker: This PR has received other commits, so Renovate will stop updating it to avoid conflicts or other problems. If you wish to abandon your changes and have Renovate start over then you can add the label rebase to this PR and Renovate will reset/recreate it.


Document taps & targets configs for users on the website

Migrated from GitLab: https://gitlab.com/meltano/meltano/-/issues/524
Originally created by @valexieva on 2019-03-22 20:03:13

Currently the tap/target config information is only in the .env files and not easily discoverable. We could expose the information on the website similarly to: https://deploy-preview-516--meltano.netlify.com/docs/plugins.html#tap-zuora
Also, the documentation for Extractors and Loaders is focused on the CLI experience and there are not instructions for how to use each from the UI. Now that the UI is becoming the more central focus, we should break out the instructions for doing things from the command line into the "Command Line Tools" section and focus the users on how to do things within the Meltano UI itself.
@bencodezen Handing this off to you, this is the vision I have for this:

create a UI-based step-by-step guide for configuration of each plugin, with screenshots (broken out into #1097 and #1098)
record a video for each one showing it being set up end-to-end and embed in the page
[x]  add link to logo on home page to take user directly to each extractor
[x]  reconsider tap-carbon on home page
[x]  create branded landing pages

Should be more focused on the benefit, put the "how it works" lower down the page
This is not documentation
Both inside the UI and outside on the website
Make sure they work for SEO purposes
Make sure they work as landing pages for ads

I think this should be pretty quick, and you've touched most of these now in your UI work. It would be awesome if you could also do this with an eye to filing UI/polish issues wherever the guide is challenging to write.

View 23 previous comments from the original issue on GitLab


Support compound timespans

This PR adds support for compound time durations.

I'm not sure how to review the integration of this PR into Jenkins, as it fails currently.


OL quits on Verilator error, but they are only warnings

Description
Running the stock spm example with OL 2023.05.19 the flow quits on error:
/foss/designs > flow.tcl -design spm
OpenLane af89aba6b7fd4c629a97fce12c26c0c8d219a947
All rights reserved. (c) 2020-2022 Efabless Corporation and contributors.
Available under the Apache License, version 2.0. See the LICENSE file for more details.

[INFO]: Using configuration in 'spm/config.json'...
[INFO]: PDK Root: /foss/pdks
[INFO]: Process Design Kit: sky130A
[INFO]: Standard Cell Library: sky130_fd_sc_hd
[INFO]: Optimization Standard Cell Library: sky130_fd_sc_hd
[INFO]: Run Directory: /foss/designs/spm/runs/RUN_2023.05.22_15.31.55
[INFO]: Saving runtime environment...
[INFO]: Preparing LEF files for the nom corner...
[INFO]: Preparing LEF files for the min corner...
[INFO]: Preparing LEF files for the max corner...
[INFO]: Running Verilator (log: spm/runs/RUN_2023.05.22_15.31.55/logs/synthesis/verilator.log)...
[ERROR]: 2 errors found by Verilator
[ERROR]: Step 0 (verilator_lint_check) failed with error:
-code 1 -level 0 -errorstack {INNER {invokeStk1 throw_error} CALL run_verilator CALL run_verilator_step CALL {run_non_interactive_mode -design spm}} -errorcode NONE -errorinfo {
    while executing
"throw_error"
    (procedure "run_verilator" line 35)
    invoked from within
"run_verilator"
    (procedure "run_verilator_step" line 3)
    invoked from within
"run_verilator_step"} -errorline 1
[INFO]: Saving current set of views in 'spm/runs/RUN_2023.05.22_15.31.55/results/final'...
[INFO]: Generating final set of reports...
[INFO]: Created manufacturability report at 'spm/runs/RUN_2023.05.22_15.31.55/reports/manufacturability.rpt'.
[INFO]: Created metrics report at 'spm/runs/RUN_2023.05.22_15.31.55/reports/metrics.csv'.
[INFO]: Saving runtime environment...
[ERROR]: Flow failed.

However, when running Verilator on the spm.v, only two warnings are thrown:
/foss/designs > iic-vlint.sh -v spm/src/spm.v
[INFO] Run verilator linting on spm/src/spm.v...
%Warning-DECLFILENAME: spm/src/spm.v:35:8: Filename 'spm' does not match MODULE name: 'TCMP'
   35 | module TCMP(clk, rst, a, s);
      |        ^~~~
                       ... For warning description see https://verilator.org/warn/DECLFILENAME?v=5.009
                       ... Use "/* verilator lint_off DECLFILENAME */" and lint_on around source to disable this message.
%Warning-UNUSEDSIGNAL: spm/src/spm.v:23:20: Signal is not driven, nor used: 'xy'
                                          : ... In instance spm
   23 |     wire[size-1:0] xy;
      |                    ^~
%Error: Exiting due to 2 warning(s)
[DONE] Bye!

@donn Please take a look. Maybe the variable QUIT_ON_VERILATOR_ERRORS should be set to 0 for some time being.
Expected Behavior
OL running spm w/o error.
Environment report
/foss/tools/openlane/2023.05 > python3 ./env.py issue-survey
Kernel: Linux v5.15.49-linuxkit
Distribution: ubuntu 22.04
Python: v3.10.6 (OK)
Critical Alert: No Docker or Docker-compatible container engine was found.
OpenLane Git Version: af89aba6b7fd4c629a97fce12c26c0c8d219a947
pip: INSTALLED
python-venv: INSTALLED
---
PDK Version Verification Status: OK
---
Git Log (Last 3 Commits)

af89aba6 2023-05-18T13:37:26+03:00 get_tag.py: Don't encode illegal docker tag when local building (#1803) - Darryl Miles -  (HEAD, tag: 2023.05.19)
2ec3bfdd 2023-05-16T21:06:51+03:00 Do not skip STA when `CLOCK_PORT` is not specified (#1797) - Kareem Farid -  (tag: 2023.05.17)
54bfaa3d 2023-05-11T21:01:58+03:00 [BOT] Update openroad_app (#1774) - Openlane Bot -  (tag: 2023.05.12)
---
Git Remotes

origin	https://github.com/The-OpenROAD-Project/OpenLane (fetch)
origin	https://github.com/The-OpenROAD-Project/OpenLane (push)

Reproduction material
Run stock spm
Relevant log output
See above.

Can you show the log file produced by OpenLane?
/foss/designs > iic-vlint.sh -v spm/src/spm.v
[INFO] Run verilator linting on spm/src/spm.v...
%Warning-DECLFILENAME: spm/src/spm.v:35:8: Filename 'spm' does not match MODULE name: 'TCMP'
   35 | module TCMP(clk, rst, a, s);
      |        ^~~~
                       ... For warning description see https://verilator.org/warn/DECLFILENAME?v=5.009
                       ... Use "/* verilator lint_off DECLFILENAME */" and lint_on around source to disable this message.
%Warning-UNUSEDSIGNAL: spm/src/spm.v:23:20: Signal is not driven, nor used: 'xy'
                                          : ... In instance spm
   23 |     wire[size-1:0] xy;
      |                    ^~
%Error: Exiting due to 2 warning(s)
[DONE] Bye!

OpenLane uses --Wno-DECLFILENAME so this can't be the output produced from OpenLane.

openlane.log

The linting output above is from our own linting script, iic-vlint.sh. I posted openlane.log above.

I meant verilator log file spm/runs/RUN_2023.05.22_15.31.55/logs/synthesis/verilator.log

verilator.log
Sorry, here the correct one.

Ah, I see. You have a WA in your Verilator build script:
# workaround for
# %Error: //build/share/verilator/include/verilated_std.sv:28:9: Redeclaring the 'std' package is not allowed
#28 | package std;
#/build/share/verilator/include/verilated_std.sv:2:1: ... note: In file included from verilated_std.sv

RUN rm /build/share/verilator/include/verilated_std.sv &&\
    touch /build/share/verilator/include/verilated_std.sv

Does this mean we also need this? Is this bricking something with Verilator? We have just one instance of Verilator for everything, not only for OL.

The interesting part here is that iic-vlint.sh didn't produce the same error. I could figure out what triggers the std error. If we can understand why this happens with OL, we can remove the WA. I think we can start by comparing iic-vlint.sh with the way OL runs it.

Here you go: https://github.com/iic-jku/iic-osic/blob/main/iic-vlint.sh
We do just a plan call verilator --lint-only -Wall "$FILE_NAME", without all the lib includes you do. Somewhere from the SKY130 libs the error seems to be triggered.

Whenever either this -I /foss/pdks/sky130A/libs.ref/sky130_fd_sc_hd/verilog/sky130_fd_sc_hd.v or -I /foss/pdks/sky130A/libs.ref/sky130_fd_sc_hd/verilog/primitives.v is included in the Verilator call it fails with the error.

I just tried. This happens with using any -I <file>.

Yes

❯ verilator --lint-only -Wall --Wno-DECLFILENAME src/spm.v -I src/spm.v
%Error: //usr/local/share/verilator/include/verilated_std.sv:28:9: Redeclaring the 'std' package is not allowed
   28 | package std;
      |         ^~~
        /usr/local/share/verilator/include/verilated_std.sv:2:1: ... note: In file included from verilated_std.sv
%Error: Exiting due to 1 error(s)

Ah, know I have a guess. The file share/verilator/include/verilated_std.sv says in the comment, it is automatic included whenever blablabla. What if it is included for every file, and of course the second time the std package is already declared.

It would be great if you did yes. Thanks.

Done, see https://github.com/verilator/verilator/issues/4215

@kareefardi The underlying Verilator issues have been fixed (see https://github.com/verilator/verilator/issues/4215), but the OL calling of Verilator needs to be fixed as well.
You call Verilator with
verilator  --lint-only  -Wall  --Wno-DECLFILENAME  --top-module spm  -I /foss/pdks/sky130A/libs.ref/sky130_fd_sc_hd/verilog/sky130_fd_sc_hd.v -I /foss/pdks/sky130A/libs.ref/sky130_fd_sc_hd/verilog/primitives.v /foss/designs/spm/src/spm.v -Wno-fatal --relative-includes
But -I shall point to directories, not files! From Verilator help:
-I<dir>                    Directory to search for includes


ci: auto build electron assets

🅰 Please add the issue ID after "Fixes #"
Fixes #

I have read the CLA Document and I hereby sign the CLA


Incorrect workloads with negative costs when grouped by workload

Environment

Seal version: main b099d71d911ab0d68a3cbd6129c7343ea8827c9d
UI version: dev-8b5d8ea

To Reproduce
Steps to reproduce the behavior:

Add a Kubernetes connector.
Create a rds serivce.

root@perf18-control-plane:/# kubectl get all -n rc1-k18
NAME                         READY   STATUS    RESTARTS      AGE
pod/mysql-817-0              1/1     Running   1 (19m ago)   7h4m
pod/mysql-clone-0            1/1     Running   1 (19m ago)   4h21m
pod/nginx-6bcd76484d-vtp9f   1/1     Running   1 (19m ago)   11h

NAME                           TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)        AGE
service/mysql-817              ClusterIP   <IP_ADDRESS>   <none>        3306/TCP       7h4m
service/mysql-817-headless     ClusterIP   None            <none>        3306/TCP       7h4m
service/mysql-clone            ClusterIP   <IP_ADDRESS>   <none>        3306/TCP       4h21m
service/mysql-clone-headless   ClusterIP   None            <none>        3306/TCP       4h21m
service/nginx                  NodePort    <IP_ADDRESS>    <none>        80:32139/TCP   11h

NAME                    READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/nginx   1/1     1            1           11h

NAME                               DESIRED   CURRENT   READY   AGE
replicaset.apps/nginx-6bcd76484d   1         1         1       11h

NAME                           READY   AGE
statefulset.apps/mysql-817     1/1     7h4m
statefulset.apps/mysql-clone   1/1     4h21m

Wait to synchronize cost data.
Go to Cluster perspective and check the Workload Costs Trend. There are some incorrect workloads with negative costs.

Check the database.

Environment

Walrus version: main 5cf1d9ef1dcb6bd27a686a24454d760fdc19ec22
UI version: dev-2c89dfb

Test Result: Cannot reproduce in latest main image.


OCPBUGS-20033: Make the OLMCatalogPlacement field immutable

Which issue(s) this PR fixes:
Fixes #OCPBUGS-20033

/hold
cc @relyt0925 @hasueki to validate you are ok enforcing this?

/hold cancel

/approve

/hold

Thanks! Dropped a question https://github.com/openshift/hypershift/pull/3113#discussion_r1371585856. Also have we tested this?

Thanks! Dropped a question #3113 (comment). Also have we tested this?

I though that the right place is the current one. It should be the other one? @enxebre

PR Updated @enxebre. Let me know if there is any other change pending.

/lgtm

/hold cancel

/cherry-pick release-4.14


ulanding_radar: move to PX4Rangefinder and cleanup

@mcsauder I don't suppose you have one of these?

@mcsauder I don't suppose you have one of these?

I do happen to!

Thanks I'll take a closer look. The goal of this exercise is to get nearly all the drivers into a reasonably consistent less bloated state and in the process understand the minor variations in start/init behavior we need to handle. Once that's complete I'd like to do another pass that makes the duplicated main/frontend code obsolete.

Specifying the serial port at start is required now. Setting default serial ports per driver doesn't make any sense, at a minimum they should be set per board. This isn't a problem for regular users that would enable the device via parameter (SENS_ULAND_CFG).

Hi @dagar , I got a chance to get back to this PR, it looks like it's functional now, (not sure if I messed up my previous test or if something got changed):
ulanding_radar start -d /dev/ttyS2
INFO  [ulanding_radar] successfully opened UART port /dev/ttyS2
nsh> INFO  [ekf2] Found range finder with instance 0
ulanding_radar status
ulanding_radar: read: 61 events, 819us elapsed, 13.43us avg, min 7us max 85us 18.371us rms
ulanding_radar: com_err: 0 events
INFO  [drivers_rangefinder] /dev/range_finder device instance: 1
 distance_sensor_s
	timestamp: 39417237  (0.510648 seconds ago)
	min_distance: 0.3150
	max_distance: 50.0000
	current_distance: 1.5000
	variance: 0.0000
	h_fov: 0.0000
	v_fov: 0.0000
	q: [0.0000, 0.0000, 0.0000, 0.0000]
	signal_quality: -1
	type: 0
	id: 0
	orientation: 25
nsh>


Support variable-length sequences for mamba block with position indices

Enable the mamba block to support variable-length sequence inputs using positional encoding.  Passing Positional Indices results in negligible performance loss for the mamba block. For common variable-length sequence distributions, performance can be improved by 4-6x.

Modification:
Refer to the link https://github.com/state-spaces/mamba/pull/244 to replace cumulative sequences with a positional encoding matrix. The position encoding is more suitable for parallel acceleration and is more commonly found in the outputs of various dataloaders.

For example, a packaged sequence of length 16 consists of four independent sub-sequences with lengths of 3, 5, 6, 2. Then corresponding:
Cumulative sequence: [0, 3, 8, 14, 16]
Position encoding:  [0, 1, 2, 0, 1, 2, 3, 4, 0, 1, 2, 3, 4, 5, 0, 1]

In the Mamba module, there are two steps that are not sequence-wise: conv1d and selective_scan. Sub-sequences within the same sequence can affect each other, and we have modified causal-conv1d and selective_scan. These two CUDA operators are implemented using position encoding to eliminate the mutual influence between sub-sequences.

How to use:

Setup two cuda_operators:

git clone --branch feat/pack_with_position_indices https://github.com/ptxu78/pack_mamba.git
python setup .../pack_mamba/setup_onlyCUDA.py install
git clone --branch feat/pack_with_position_indices https://github.com/ptxu78/causal-conv1d-pack.git
python setup .../causal-conv1d-pack/setup_onlyCUDA.py install

Pack the variable-length sequences and input them, along with the corresponding position encoding.

This PR attempts to resolve the issue derived from training with packed data?

This PR attempts to resolve the issue derived from training with packed data?

Yes, this PR allow Mamba to handle packed data more effectively: it significantly increases throughput while ensuring the mathematical equivalence of the training results.


in azure - setting search domains causes the server to become inaccessible

Environment info

Version: 3.0
Deployment: AZURE
Customer: none

Actual behavior

after updating the search domains in an azure server, it becomes inaccessible through the network
this could be a result of performing systemctl restart network in `os_utils._set_search_domains - when I tried to run the command in the shell I get the same result (no connectivity)
I also got no connectivity a few seconds after updating DOMAIN= in /etc/sysconfig/network-scripts/ifcfg-eth0. not sure if it's a coincident or some script running in azure

Expected behavior

we should either make it work or disable Search Domains updating in azure

Steps to reproduce

update search domain in DNS configurations

Screenshots or Logs or other output that would be helpful

No longer relevant due to OVA deployment support removal


New 1.0.0 Visualizer

Implement an updated, modernized account visualizer for the 1.x API console.

https://github.com/reaviz/reagraph