[SPARK-23707][SQL] Don't need shuffle exchange with single partition for 'spark.range'

What changes were proposed in this pull request?
Just like #20726. There is no need 'Exchange' when spark.range produce only one partition.
How was this patch tested?
New UT.

@cloud-fan pls take a look, this is a small change. Thanks a lot.

Can one of the admins verify this patch?

Can one of the admins verify this patch?

@cloud-fan, pls take a look, thanks a lot.

ok to test

Test build #88474 has started for PR 20844 at commit b52703c.

Actually, range is mostly for testing. I don't think we should do too match logical optimization for it, like avoid shuffle/sort. cc @hvanhovell

Test build #88474 has finished for PR 20844 at commit b52703c.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88474/
Test FAILed.

This change is very simple, and just make it consistent with other LeafNode.

Test build #88500 has started for PR 20844 at commit 15165d7.

Test build #88500 has finished for PR 20844 at commit 15165d7.

This patch fails due to an unknown error code, -9.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88500/
Test FAILed.

Can one of the admins verify this patch?

I think it has been fixed?

Yup, the regression tests pass at the very least.

thanks for all. Closes it.


[kube-state-metrics] kubernetes_state.job.succeeded/failed wildly out of wack

Output of the info page (if this is a bug)
==============
Agent (v6.15.0)
===============

  Status date: 2019-12-06 16:34:47.824750 UTC
  Agent start: 2019-11-15 03:36:36.073832 UTC
  Pid: 341
  Go Version: go1.12.9
  Python Version: 2.7.16
  Check Runners: 4
  Log Level: WARNING

  Paths
  =====
    Config File: /etc/datadog-agent/datadog.yaml
    conf.d: /etc/datadog-agent/conf.d
    checks.d: /etc/datadog-agent/checks.d

  Clocks
  ======
    System UTC time: 2019-12-06 16:34:47.824750 UTC

  Host Info
  =========
    bootTime: 2019-11-15 03:34:15.000000 UTC
    kernelVersion: 4.19.68-coreos
    os: linux
    platform: debian
    platformFamily: debian
    platformVersion: 10.1
    procs: 71
    uptime: 2m23s
    virtualizationRole: guest
    virtualizationSystem: xen

  Hostnames
  =========
    ec2-hostname: ip-172-16-207-57.us-west-2.compute.internal
    host_aliases: [ip-172-16-207-57.us-west-2.compute.internal]
    hostname: i-00fc1de3f7cbadca3
    instance-id: i-00fc1de3f7cbadca3
    socket-fqdn: dd-prod-datadog-szzcd
    socket-hostname: dd-prod-datadog-szzcd
    host tags:
      [kube_cloud_provider:aws
      kube_cluster_id:steel
      kube_env:prod
      kube_region:us-west-2]
    hostname provider: aws
    unused hostname providers:
      configuration/environment: hostname is empty
      gce: unable to retrieve hostname from GCE: status code 404 trying to GET http://<IP_ADDRESS>/computeMetadata/v1/instance/hostname

=========
Collector
=========

  Running Checks
  ==============

    coredns (1.2.0)
    ---------------
      Instance ID: coredns:eab009ef2bf6f24c [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/coredns.d/auto_conf.yaml
      Total Runs: 124,073
      Metric Samples: Last Run: 488, Total: 1 M
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 1, Total: 124,073
      Average Execution Time : 44ms

    cpu
    ---
      Instance ID: cpu [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/cpu.d/conf.yaml.default
      Total Runs: 124,073
      Metric Samples: Last Run: 6, Total: 744,432
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 0, Total: 0
      Average Execution Time : 0s

    cri
    ---
      Instance ID: cri [ERROR]
      Configuration Source: file:/etc/datadog-agent/conf.d/cri.d/conf.yaml.default
      Total Runs: 124,072
      Metric Samples: Last Run: 0, Total: 0
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 0, Total: 0
      Average Execution Time : 0s
      Error: permanent failure in criutil: retry number exceeded
      No traceback
      Warning: Error initialising check: permanent failure in criutil: retry number exceeded

    disk (2.5.1)
    ------------
      Instance ID: disk:e5dffb8bef24336f [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/disk.d/conf.yaml.default
      Total Runs: 124,073
      Metric Samples: Last Run: 208, Total: 1 M
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 0, Total: 0
      Average Execution Time : 117ms

    file_handle
    -----------
      Instance ID: file_handle [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/file_handle.d/conf.yaml.default
      Total Runs: 124,072
      Metric Samples: Last Run: 5, Total: 620,360
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 0, Total: 0
      Average Execution Time : 0s

    io
    --
      Instance ID: io [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/io.d/conf.yaml.default
      Total Runs: 124,073
      Metric Samples: Last Run: 156, Total: 1 M
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 0, Total: 0
      Average Execution Time : 1ms

    kubelet (3.3.4)
    ---------------
      Instance ID: kubelet:d884b5186b651429 [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/kubelet.d/conf.yaml.default
      Total Runs: 124,072
      Metric Samples: Last Run: 2,363, Total: 1 M
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 4, Total: 496,288
      Average Execution Time : 1.091s

    kubernetes_state (4.7.1)
    ------------------------
      Instance ID: kubernetes_state:853b0bed6c1a5803 [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/kubernetes_state.d/auto_conf.yaml
      Total Runs: 124,031
      Metric Samples: Last Run: 31,756, Total: 1 M
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 1,406, Total: 1 M
      Average Execution Time : 2.677s

    load
    ----
      Instance ID: load [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/load.d/conf.yaml.default
      Total Runs: 124,073
      Metric Samples: Last Run: 6, Total: 744,438
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 0, Total: 0
      Average Execution Time : 0s

    memory
    ------
      Instance ID: memory [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/memory.d/conf.yaml.default
      Total Runs: 124,072
      Metric Samples: Last Run: 17, Total: 1 M
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 0, Total: 0
      Average Execution Time : 0s

    network (1.11.5)
    ----------------
      Instance ID: network:e0204ad63d43c949 [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/network.d/conf.yaml.default
      Total Runs: 124,073
      Metric Samples: Last Run: 505, Total: 1 M
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 0, Total: 0
      Average Execution Time : 11ms

    ntp
    ---
      Instance ID: ntp:d884b5186b651429 [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/ntp.d/conf.yaml.default
      Total Runs: 2,068
      Metric Samples: Last Run: 0, Total: 0
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 1, Total: 2,068
      Average Execution Time : 20.013s

    uptime
    ------
      Instance ID: uptime [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/uptime.d/conf.yaml.default
      Total Runs: 124,072
      Metric Samples: Last Run: 1, Total: 124,072
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 0, Total: 0
      Average Execution Time : 0s

========
JMXFetch
========

  Initialized checks
  ==================
    no checks

  Failed checks
  =============
    no checks

=========
Forwarder
=========

  Transactions
  ============
    CheckRunsV1: 124,072
    Dropped: 0
    DroppedOnInput: 0
    Events: 0
    HostMetadata: 0
    IntakeV1: 13,441
    Metadata: 0
    Requeued: 5
    Retried: 5
    RetryQueueSize: 0
    Series: 0
    ServiceChecks: 0
    SketchSeries: 0
    Success: 706,682
    TimeseriesV1: 124,072

  Transaction Errors
  ==================
    Total number: 1
    Errors By Type:

  API Keys status
  ===============
    API key ending with 2c443: API Key valid

==========
Endpoints
==========
  https://app.datadoghq.com - API Key ending with:
      - 2c443

==========
Logs Agent
==========

  Logs Agent is not running

=========
Aggregator
=========
  Checks Metric Sample: 4 G
  Dogstatsd Metric Sample: 310,182
  Event: 1
  Events Flushed: 1
  Number Of Flushes: 124,072
  Series Flushed: 4 G
  Service Check: 173.1 M
  Service Checks Flushed: 173.2 M

=========
DogStatsD
=========
  Event Packets: 0
  Event Parse Errors: 0
  Metric Packets: 310,181
  Metric Parse Errors: 0
  Service Check Packets: 0
  Service Check Parse Errors: 0
  Udp Bytes: 17.2 M
  Udp Packet Reading Errors: 0
  Udp Packets: 310,182
  Uds Bytes: 0
  Uds Origin Detection Errors: 0
  Uds Packet Reading Errors: 0
  Uds Packets: 0

It should be noted that this particular agent is scraping only part of the metrics, because of dd agent issues we've divided our KSM into two pods one for all pod metrics and one for all other metrics. They are located on different nodes and therefore scraped by different agents.
Describe what happened:
Kubernetes 1.16.2
KSM 1.18.0
DD-agent 6.15.0
failed/succeeded job counts do not line up at all.

I'm not sure what explains any metrics at all flowing, i would expect a fairly binary outcome here.
Describe what you expected:
I expect my stats to line up.
It was mentioned by @CharlyF that potentially the problem here is that you haven't run any tests against KSM 1.18.0. This is quite possibly the problem I bisecting through KSM to see if I can't quick patch this.
Here are my expectations, I would like you guys to do a better job publishing tested versions of major components for each integration. I'm growing frustrated with the firedrill of datadog issues whenever we perform an upgrade.
Just a few weeks back I found this bug https://github.com/DataDog/integrations-core/issues/4923.
The post-mortem timeline for that is:

02/26/19 Deprication of pod/container_name in the kubelet is announced (as far as I can tell) in the 1.14.0-beta.1 patch notes. At this point both the new pod/container labels as well as the old pod_name/container_name labels are on the metrics produced from the kubelet.
09/07/19 Kube 1.16.0-Alpha.3 is released in the patch notes it's made clear that the labels are now fully depricated.
09/18/19 Kube 1.16.0 is released
10/29/19 I create the issue after discovering that the kubelet integration was still looking for the old metric labels.
10/30/19 To your credit your team acted quickly to get a fix in place
11/05/19 That fix makes it to an agent (6.15.0)

Kubernetes is moving fast but you had 9 months to react.
Steps to reproduce the issue:
Run the above noted configuration.
Additional environment details (Operating System, Cloud provider, etc):
NA

Thanks for opening this issue @SleepyBrett.
The best way forward is to verify that the values you expect are properly listed in the KSM output, our source of truth. As you mentioned:

It should be noted that this particular agent is scraping only part of the metrics, because of dd agent issues we've divided our KSM into two pods one for all pod metrics and one for all other metrics. They are located on different nodes and therefore scraped by different agents.

Could you share more details on the issues you had with the Datadog Agent ? Are we talking large memory footprint or something else ?
With regards to how you separated the KSM metrics, do you mean splitting the collectors or are you also using the autosharding feature ?
Could you exec in the Agent collecting the jobs metrics (I am assuming you just split the collectors) and curl the ksm endpoint ? I am specially interested in the kube_job_failed and kube_job_status_failed metrics (two different metrics in our check, but I am not familiar with how Prometheus evaluates both and from your screenshot it seems like only the status_failed is plotted).
Best,
.C

Large memory footprints and timeouts related to trying to process a whole KSM, the sharding was actually potentially your idea (at least your team's idea) (https://github.com/DataDog/datadog-agent/issues/1665, https://github.com/DataDog/datadog-agent/issues/1853)... though we were also working with a sales support engineer on this issue.
We had no access to the autosharding feature at the time so we sharded by collector (pods in one pod, everything else in the other). I did notice the addition to a beta feature for sharding and have it on a list to try out.
...
kube_job_failed{namespace="cvp",job_name="nonprod-automl-deployer-1575657000",condition="true"} 1
kube_job_failed{namespace="cvp",job_name="nonprod-automl-deployer-1575657000",condition="false"} 0
kube_job_failed{namespace="cvp",job_name="nonprod-automl-deployer-1575657000",condition="unknown"} 0
...
kube_job_status_failed{namespace="devportal",job_name="scraper-1575547200"} 0
kube_job_status_failed{namespace="dsdatafnd",job_name="no-meta-gp-zb4tdtp-delete-job"} 1
...

enough for you?

@SleepyBrett Just giving a heads up that the fix for cronjobs was released with v6.16.x of the agent via https://github.com/DataDog/integrations-core/pull/4943, so if upgrading is a possibility that should provide a first step.


Need Help: How to set an event weekly to the past and to the future?

At the moment, the minimum date is now, but I want to add an event to the past and to the future without any limit. How can I do this?
new RRule({
            freq: RRule.WEEKLY,
            byweekday: RRule[weekDayName.substr(0, 2).toUpperCase()],
          })

Using dtstart with a date at 1970-01-01 has a problem with timezones with daylight time.

I've found that doing rrule.options.dtstart = new Date("1970-01-01"); should get you somewhat closer; it seems to default the dtstart to today.


Small emulation enhancements

These are some small quality of life improvements and clean ups.
Delete a redundant class (TransformedModelConfig) and kill the training if the loss value is NaN. See commits for more motivation.

Switched to using a tensorflow callback to kill the training. Here is what the output looks like:
Epoch 1/2
Batch 1: Invalid loss, terminating training
2/2 - 2s - loss: nan - cloud_water_mixing_ratio_after_precpd_loss: nan - specific_humidity_after_precpd_loss: nan - total_precipitation_loss: nan - air_temperature_after_precpd_loss: nan


#20 Attempt to reconnect after ping fails

My naive approach to attempt to fix #20.
When building the local environment, I found that some commands in Makefile do not use virtual environment managed by poetry. I have included the changes to make these commands runnable on my local machine, but if it's breaking CI do let me know and I'll remove them in the merge request.
I did not include tests because I do not know if there is a reliable way to produce the "stale" connection in ClickHouse other than opening the connection pool without doing anything for an hour.

@long2ice I ran into the same problem and it looks like @stdc105 (#20) and @harshitsinghai77 (#25) also ran into it. How are you doing with this PR? Perhaps I can do something to help us get rid of this problem faster?


[LMS-331] sample LMS data for assignments & submissions

sample LMS data to support importing of assignments and submissions.  assignments.csv, users.csv, sections.csv, section-assoc.csv, submissions.csv

We need to re-organize the files so that they fit into the file system properly, including splitting into separate section files.

Moving to a branch in the main repo


Upgrade shoulda-matchers to 3.0.0.rc1

In anticipation of the 3.0.0 release of shoulda-matchers gem.

Thanks @leesmith, much appreciated :+1:


Can't bind to 'items' since it isn't a known property of 'select-searchable'

Template parse errors:
Can't bind to 'items' since it isn't a known property of 'select-searchable'.
1. If 'select-searchable' is an Angular component and it has 'items' input, then verify that it is part of this module.
2. If 'select-searchable' is a Web Component then add 'CUSTOM_ELEMENTS_SCHEMA' to the<EMAIL_ADDRESS>of this component to suppress this message.
3. To allow any property add 'NO_ERRORS_SCHEMA' to the<EMAIL_ADDRESS>of this component. ("
							itemValueField="id"
							itemTextField="name"
							[ERROR ->][items]="ports"
							[canSearch]="true"
							(onChange)="portChange($event)">
"): ng:///HiremePageModule/HiremePage.html@46:7
Can't bind to 'canSearch' since it isn't a known property of 'select-searchable'.
1. If 'select-searchable' is an Angular component and it has 'canSearch' input, then verify that it is part of this module.
2. If 'select-searchable' is a Web Component then add 'CUSTOM_ELEMENTS_SCHEMA' to the<EMAIL_ADDRESS>of this component to suppress this message.
3. To allow any property add 'NO_ERRORS_SCHEMA' to the<EMAIL_ADDRESS>of this component. ("
							itemTextField="name"
							[items]="ports"
							[ERROR ->][canSearch]="true"
							(onChange)="portChange($event)">
						</select-searchable>
"): ng:///HiremePageModule/HiremePage.html@47:7
'select-searchable' is not a known element:
1. If 'select-searchable' is an Angular component, then verify that it is part of this module.
2. If 'select-searchable' is a Web Component then add 'CUSTOM_ELEMENTS_SCHEMA' to the<EMAIL_ADDRESS>of this component to suppress this message. ("
					</ion-item-divider>
					<ion-item>
						[ERROR ->]<select-searchable
							[(ngModel)]="port3"
							title="Port"
"): ng:///HiremePageModule/HiremePage.html@41:6

`Error: Template parse errors:
Can't bind to 'items' since it isn't a known property of 'select-searchable'.

If 'select-searchable' is an Angular component and it has 'items' input, then verify that it is part of this module.
If 'select-searchable' is a Web Component then add 'CUSTOM_ELEMENTS_SCHEMA' to the<EMAIL_ADDRESS>of this component to suppress this message.
To allow any property add 'NO_ERRORS_SCHEMA' to the<EMAIL_ADDRESS>of this component. ("
itemValueField="id"
itemTextField="name"
[ERROR ->][items]="ports"
[canSearch]="true"
(onChange)="portChange($event)">
"): ng:///HiremePageModule/HiremePage.html@46:7
Can't bind to 'canSearch' since it isn't a known property of 'select-searchable'.
If 'select-searchable' is an Angular component and it has 'canSearch' input, then verify that it is part of this module.
If 'select-searchable' is a Web Component then add 'CUSTOM_ELEMENTS_SCHEMA' to the<EMAIL_ADDRESS>of this component to suppress this message.
To allow any property add 'NO_ERRORS_SCHEMA' to the<EMAIL_ADDRESS>of this component. ("
itemTextField="name"
[items]="ports"
[ERROR ->][canSearch]="true"
(onChange)="portChange($event)">

"): ng:///HiremePageModule/HiremePage.html@47:7
'select-searchable' is not a known element:
If 'select-searchable' is an Angular component, then verify that it is part of this module.
If 'select-searchable' is a Web Component then add 'CUSTOM_ELEMENTS_SCHEMA' to the<EMAIL_ADDRESS>of this component to suppress this message. ("

[ERROR ->]<select-searchable
[(ngModel)]="port3"
title="Port"
"): ng:///HiremePageModule/HiremePage.html@41:6
at syntaxError (http://localhost:8100/build/vendor.js:82258:34)
at TemplateParser.parse (http://localhost:8100/build/vendor.js:106121:19)
at JitCompiler._parseTemplate (http://localhost:8100/build/vendor.js:115549:37)
at JitCompiler._compileTemplate (http://localhost:8100/build/vendor.js:115524:23)
at http://localhost:8100/build/vendor.js:115426:62
at Set.forEach ()
at JitCompiler._compileComponents (http://localhost:8100/build/vendor.js:115426:19)
at http://localhost:8100/build/vendor.js:115296:19
at Object.then (http://localhost:8100/build/vendor.js:82247:77)
at JitCompiler._compileModuleAndComponents (http://localhost:8100/build/vendor.js:115295:26)`

Ionic Version:

Ionic Framework: 3.9.2
Ionic App Scripts: 3.1.8
Angular Core: 5.0.3
Angular Compiler CLI: 5.0.3
Node: 8.4.0
OS Platform: Windows 7
Navigator Platform: Win32
User Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36

Hi, you are probably using lazy loading. Have a look here:
https://github.com/eakoriakin/ionic-select-searchable#1-why-do-i-get-error-cant-bind-to-items-since-it-isnt-a-known-property-of-select-searchable

Then check that you've imported SelectSearchableModule to your main AppModule:
import { NgModule } from '@angular/core';
import { SelectSearchableModule } from 'ionic-select-searchable';

@NgModule({
    ...
    imports: [
        ...
        SelectSearchableModule
    ],
    ...
})
export class AppModule { }

@biswadeep-sarkar have a look here as well:
https://github.com/eakoriakin/ionic-select-searchable-demo/issues/12#issuecomment-377646075

Sorry, i have the same issue in ionic 3 but using the main example. I have all the imports fine. I tried to follow the link below but it was removed.

Check this link:
https://github.com/eakoriakin/ionic-selectable/wiki#why-do-i-get-error-cant-bind-to-items-since-it-isnt-a-known-property-of-ionic-selectable

Thanks! Is strange, it show me a error so i removed code and started again, compiled with : ionic cordova run android --device -l -c , and it show me the view with control, but nothing happens. Even, I see if it show me the console log when load the change event but no. Now, I started again the app on my mobile and it show me again the error : "Error: No provider for SelectSearchableComponent!" But i tried to fix all the things .Im using ionic 3, im not sure if maybe is the version of component

Hi Evegnii
Now I've a similar issue. I'm using ionic-selectable in my project in more pages. This pages are used as Components for
ModalController and they are working fine.
Now I want to use ionic-selectable in a normal page an I get the error:
core.js:10069 NG0303: Can't bind to 'items' since it isn't a known property of 'ion-selectable'.
https://github.com/petervarkoly/cranix-web/blob/d60c3f8037cd3b62d77a4999f36ddf60b6011b0b/src/app/protected/edu/lessons/tabs/room-control/room-control.component.html#L43
Do you have a hint for me?


Zip code change input

Hey guys, first off, exceptional job with this library, huge props!
Quick enhancement, with regards to zip code entry when this extra is enabled:
scanIntent.putExtra(CardIOActivity.EXTRA_REQUIRE_POSTAL_CODE, true);
You get to the screen where you enter in the zip code and it switches to letters instead of numbers for the edit text.
Would it make sense to change the input method on the zip code edit text to:
android:inputType="number"
Or is that something done intentionally due to other places using this may have something other than numbers in their zip codes?

Its generic, there are alphanumeric zip codes, https://en.wikipedia.org/wiki/List_of_postal_codes

Understood, ty for clarifying.


Use ng2-ui-auth from dockstore org

Description
We forked and modified ng2-ui-auth to work with newer versions of Angular here. This PR uses that package directly from GitHub, saving us the need to publish on npm.
Review Instructions
Describe if this ticket needs review and if so, how one may go about it in qa and/or staging environments.
For example, a ticket based on Security Hub, Snyk, or Dependabot may not need review since those services
will generate new warnings if the issue has not been resolved properly. On the other hand, an infrastructure
ticket that results in visible changes to the end-user will definitely require review.
Many tickets will likely be between these two extremes, so some judgement may be required.
Issue
A link to a github issue or SEAB- ticket (using that as a prefix)
Security
If there are any concerns that require extra attention from the security team, highlight them here.
Please make sure that you've checked the following before submitting your pull request. Thanks!

[ ] Check that your code compiles by running npm run build
[ ] Ensure that the PR targets the correct branch. Check the milestone or fix version of the ticket.
[ ] If this is the first time you're submitting a PR or even if you just need a refresher, consider reviewing our style guide
[ ] Do not bypass Angular sanitization (bypassSecurityTrustHtml, etc.), or justify why you need to do so
[ ] If displaying markdown, use the markdown-wrapper component, which does extra sanitization
[ ] Do not use cookies, although this may change in the future
[ ] Run npm audit and ensure you are not introducing new vulnerabilities
[ ] Do due diligence on new 3rd party libraries, checking for CVEs
[ ] Don't allow user-uploaded images to be served from the Dockstore domain
[ ] If this PR is for a user-facing feature, create and link a documentation ticket for this feature (usually in the same milestone as the linked issue). Style points if you create a documentation PR directly and link that instead.
[ ] Check whether this PR disables tests. If it legitimately needs to disable a test, create a new ticket to re-enable it in a specific milestone.

Not doing this, comment.


Clean up

first pass.
removed django 1.4-1.6

Don't forget to update the version in the README.

thanks for pointing :D updating

updated

Thanks @auvipy!

well come :+1:  check this one https://github.com/alex/django-taggit/pull/387 its passing


Backend Development

Deployed Website
Lesson on Boolean and If-Else Statements
Focus and Habits

+2.7/3
Riya did all of the work for this week:

She deployed her website on (riyacsa.tk)
She completed the lesson on Boolean and If-Else Statements
She took notes on the Harvard video


Add message about django-suit integration

We added some templates to make the admin look nice with django-suit. I'm not sure if this is something you'd be interesting in including or not, so I added README. If you do want to include it, but have a better place, let me know and I can move it.

More documentation is always good :). Thanks for the PR!


[Proposal] URIs and numbers linked only in preview

My motivation here is that currently these strings can not be edited in Noto after the note is reopened. On trying to set the cursor inside those elements, the "open with" panel pops up and the keyboard vanishes. Even workarounds don't work, namely cutting and re-pasting elsewhere, even into a new note: both yields a linked/non-editable URI.

Yeah, this is going to be the default behavior in 2.2.0, just how it's used to be before. Making links clickable in the editor caused this issue #101. And it's already been fixed by disabling links in the editor.

And it's already been fixed by disabling links in the editor.

Great, thanks!


Post not shown in Subscribed channel, which is created by the owner

Create a Channel.
Subscribe / Unsubscribe, there is another bug: you can still subscribe channel created by yourself, and there are 2 duplicated channels listed in the left Navbar
Submit a post.
back to home and the post is not shown in u subscribed channel, but it listed in all channels

I tested this on my machine and a few others, could reproduce the issue.


Allow to build VST3 for plugins with external UI

PR fixes this compilation issue:
In file included from ../../distrho/DistrhoUIMain.cpp:30:
../../distrho/src/DistrhoUIVST3.cpp: In member function ‘v3_result DISTRHO::UIVst3::onKeyDown(int16_t, int16_t, int16_t)’:
../../distrho/src/DistrhoUIVST3.cpp:222:20: error: ‘class DISTRHO::UIExporter’ has no member named ‘handlePluginKeyboardVST3’
  222 |         return fUI.handlePluginKeyboardVST3(true, static_cast<uint>(keychar), dglcode, dglmods) ? V3_TRUE : V3_FALSE;
      |                    ^~~~~~~~~~~~~~~~~~~~~~~~
../../distrho/src/DistrhoUIVST3.cpp: In member function ‘v3_result DISTRHO::UIVst3::onKeyUp(int16_t, int16_t, int16_t)’:
../../distrho/src/DistrhoUIVST3.cpp:253:20: error: ‘class DISTRHO::UIExporter’ has no member named ‘handlePluginKeyboardVST3’
  253 |         return fUI.handlePluginKeyboardVST3(false, static_cast<uint>(keychar), dglcode, dglmods) ? V3_TRUE : V3_FALSE;
      |                    ^~~~~~~~~~~~~~~~~~~~~~~~
make: *** [../../Makefile.plugins.mk:302: ../../build/d_embedextui/DistrhoUIMain_VST3.cpp.o] Error 1

Oh I forgot about this one, thanks!
But.. since the external UI won't receive those events, would be best to just ifdef the UIVST3 stuff entirely, and return not-implemented from the C call directly, skipping the C++ cast.
btw, sorta related to this...
as typical with stuff steinberg does, no one cares so I have yet to see a host that sends out these events :sweat_smile:
this is why it is incomplete, I am unable to test the plugin part...

I am not sure to fully understand what is the root issue. In the case of the webview UI, it already seems to work on Mac/VST3. There are some bugs but the native Cocoa widget WKWebView renders and responds to user events as expected. Still need to spend time on the bugs. Naturally this is just a single case, likely missing something here...
CADisplayLink  might be useful for the idle timer according to https://stackoverflow.com/questions/30856330/trying-to-create-an-accurate-timer-with-gcd and https://dmtopolog.com/cadisplaylink-and-its-applications/
In the case of Windows/VST3 yes, the web view does not even render.  I am not really a Windows programmer so not sure where to look, but this sounds useful https://docs.microsoft.com/en-us/windows/win32/winmsg/window-features#message-only-windows . The webview project does add an invisible window with  GWLP_USERDATA to the host provided hwnd for some purposes.  Quite sure this is not new at all :)


Modernize OpenVINO-based Nuclio functions and allow them to run on Kubernetes

Motivation and context

Currently, OpenVINO-based functions assume that a local directory will be mounted into the container. In Kubernetes, that isn't possible, so implement an alternate approach: create a separate base image and inherit the function image from it.
In addition, implement some modernizations:

Upgrade the version of OpenVINO to the latest (2022.3). Make the necessary updates to the code. Note that 2022.1 introduced an entirely new inference API, but I haven't switched to it yet to minimize changes.

Use the runtime version of the Docker image as the base instead of the dev version. This significantly reduces the size of the final image (by ~3GB).

Replace the faster_rcnn_inception_v2_coco model with faster_rcnn_inception_resnet_v2_atrous_coco, as the former has been
removed from OMZ.

Ditto with person-reidentification-retail-0300 -> 0277.

The IRs used in the DEXTR function are not supported by OpenVINO anymore (format too old), so rewrite the build process to create them from the original code/weights instead.

How has this been tested?

I manually tried each affected function to make sure they still work.
Checklist

[x] I submit my changes into the develop branch
[ ] I have added a description of my changes into the CHANGELOG file
[ ] I have updated the documentation accordingly
[ ] I have added tests to cover my changes
[ ] I have linked related issues (see GitHub docs)
[ ] I have increased versions of npm packages if it is necessary
(cvat-canvas,
cvat-core,
cvat-data and
cvat-ui)

License

[x] I submit my code changes under the same MIT License that covers the project.
Feel free to contact the maintainers if that's a concern.

Converting to draft until I figure out the Helm pipeline failures.

./deploy_cpu.sh openvino/omz/intel/face-detection-0205/nuclio/ causes:
23.05.12 04:13:31.372 cessor.healthcheck.server (I) Listening {"listenAddress": ":8082"}
23.05.12 04:13:31.372            processor.http (D) Creating worker pool {"num": 2}
23.05.12 04:13:31.372 sor.http.w0.python.logger (D) Creating listener socket {"path": "/tmp/nuclio-rpc-chevaaucno8m37gurf20.sock"}
23.05.12 04:13:31.372 sor.http.w1.python.logger (D) Creating listener socket {"path": "/tmp/nuclio-rpc-chevaaucno8m37gurf1g.sock"}
23.05.12 04:13:31.372 sor.http.w0.python.logger (W) Python 3.6 runtime is deprecated and will soon not be supported. Please migrate your code and use Python 3.7 runtime (`python:3.7`) or higher
23.05.12 04:13:31.372 sor.http.w0.python.logger (D) Using Python wrapper script path {"path": "/opt/nuclio/_nuclio_wrapper.py"}
23.05.12 04:13:31.372 sor.http.w0.python.logger (D) Using Python handler {"handler": "main:handler"}
23.05.12 04:13:31.372 sor.http.w1.python.logger (W) Python 3.6 runtime is deprecated and will soon not be supported. Please migrate your code and use Python 3.7 runtime (`python:3.7`) or higher
23.05.12 04:13:31.372 sor.http.w1.python.logger (D) Using Python wrapper script path {"path": "/opt/nuclio/_nuclio_wrapper.py"}
23.05.12 04:13:31.372 sor.http.w1.python.logger (D) Using Python handler {"handler": "main:handler"}
23.05.12 04:13:31.372 sor.http.w1.python.logger (E) Can't find Python exe {"error": "exec: \"/opt/nuclio/common/openvino/python3\": stat /opt/nuclio/common/openvino/python3: no such file or directory"}
23.05.12 04:13:31.372 sor.http.w0.python.logger (E) Can't find Python exe {"error": "exec: \"/opt/nuclio/common/openvino/python3\": stat /opt/nuclio/common/openvino/python3: no such file or directory"}

Error - exec: "/opt/nuclio/common/openvino/python3": stat /opt/nuclio/common/openvino/python3: no such file or directory
    ...//nuclio/pkg/processor/runtime/rpc/abstract.go:239

Call stack:
Can't run wrapper
    ...//nuclio/pkg/processor/runtime/rpc/abstract.go:239
Failed to run wrapper
    ...//nuclio/pkg/processor/runtime/rpc/abstract.go:106
Failed to start runtime
    /nuclio/pkg/processor/worker/factory.go:100
Failed to create worker
    /nuclio/pkg/processor/worker/factory.go:119
Failed to create workers
    /nuclio/pkg/processor/worker/factory.go:129

    /nuclio/pkg/platform/local/platform.go:1168
Failed to deploy function
    ...//nuclio/pkg/platform/abstract/platform.go:197
  NAMESPACE |               NAME               | PROJECT |  STATE   | REPLICAS | NODE PORT
  nuclio    | openvino-dextr                   | cvat    | ready    | 1/1      |     32770
  nuclio    | openvino-omz-face-detection-0205 | cvat    | building | 1/1      |
  nuclio    | pth-facebookresearch-sam-vit-h   | cvat    | ready    | 1/1      |     32769

./deploy_cpu.sh openvino/dextr/nuclio/ causes another error:
Step 10/15 : ADD export.py adaptive-pool.patch .
When using ADD with more than one source file, the destination must be a directory and end with a /
Deploying openvino/dextr function...
23.05.12 11:16:41.155                     nuctl (I) Deploying function {"name": ""}
23.05.12 11:16:41.155                     nuctl (I) Building {"builderKind": "docker", "versionInfo": "Label: 1.8.14, Git commit: cbb0774230996a3eb4621c1a2079e2317578005b, OS: linux, Arch: amd64, Go version: go1.17.8", "name": ""}
23.05.12 11:16:41.553                     nuctl (I) Staging files and preparing base images
23.05.12 11:16:41.554                     nuctl (W) Python 3.6 runtime is deprecated and will soon not be supported. Please migrate your code and use Python 3.7 runtime (`python:3.7`) or higher
23.05.12 11:16:41.555                     nuctl (I) Building processor image {"registryURL": "", "imageName": "cvat.openvino.dextr:latest"}
23.05.12 11:16:41.555     nuctl.platform.docker (I) Pulling image {"imageName": "quay.io/nuclio/handler-builder-python-onbuild:1.8.14-amd64"}
23.05.12 11:16:44.556     nuctl.platform.docker (W) Docker command outputted to stderr - this may result in errors {"workingDir": "/tmp/nuclio-build-1967427100/staging", "cmd": "docker build --network host --force-rm -t nuclio-onbuild-chevbr12gbapuflbhibg -f /tmp/nuclio-build-1967427100/staging/Dockerfile.onbuild   --build-arg NUCLIO_LABEL=1.8.14 --build-arg NUCLIO_ARCH=amd64 --build-arg NUCLIO_BUILD_LOCAL_HANDLER_DIR=handler  .", "stderr": "DEPRECATED: The legacy builder is deprecated and will be removed in a future release.\n            Install the buildx component to build images with BuildKit:\n            https://docs.docker.com/go/buildx/\n\n"}
23.05.12 11:16:45.236     nuctl.platform.docker (I) Pulling image {"imageName": "quay.io/nuclio/uhttpc:0.0.1-amd64"}
23.05.12 11:16:50.682     nuctl.platform.docker (W) Docker command outputted to stderr - this may result in errors {"workingDir": "/tmp/nuclio-build-1967427100/staging", "cmd": "docker build --network host --force-rm -t nuclio-onbuild-chevbsh2gbapuflbhic0 -f /tmp/nuclio-build-1967427100/staging/Dockerfile.onbuild   --build-arg NUCLIO_LABEL=1.8.14 --build-arg NUCLIO_ARCH=amd64 --build-arg NUCLIO_BUILD_LOCAL_HANDLER_DIR=handler  .", "stderr": "DEPRECATED: The legacy builder is deprecated and will be removed in a future release.\n            Install the buildx component to build images with BuildKit:\n            https://docs.docker.com/go/buildx/\n\n"}
23.05.12 11:16:50.992            nuctl.platform (I) Building docker image {"image": "cvat.openvino.dextr:latest"}
23.05.12 11:16:54.636     nuctl.platform.docker (W) Docker command outputted to stderr - this may result in errors {"workingDir": "/tmp/nuclio-build-1967427100/staging", "cmd": "docker build --network host --force-rm -t cvat.openvino.dextr:latest -f /tmp/nuclio-build-1967427100/staging/Dockerfile.processor   --build-arg NUCLIO_LABEL=1.8.14 --build-arg NUCLIO_ARCH=amd64 --build-arg NUCLIO_BUILD_LOCAL_HANDLER_DIR=handler  .", "stderr": "DEPRECATED: The legacy builder is deprecated and will be removed in a future release.\n            Install the buildx component to build images with BuildKit:\n            https://docs.docker.com/go/buildx/\n\npull access denied for cvat.openvino.dextr.base, repository does not exist or may require 'docker login': denied: requested access to the resource is denied\n"}
23.05.12 11:16:54.640                     nuctl (W) Failed to create a function; setting the function status {"err": "Failed to build processor image", "errVerbose": "\nError - exit status 1\n    /nuclio/pkg/cmdrunner/shellrunner.go:96\n\nCall stack:\nstdout:\nSending build context to Docker daemon  50.08MB\r\r\nStep 1/12 : FROM cvat.openvino.dextr.base\n\nstderr:\nDEPRECATED: The legacy builder is deprecated and will be removed in a future release.\n            Install the buildx component to build images with BuildKit:\n            https://docs.docker.com/go/buildx/\n\npull access denied for cvat.openvino.dextr.base, repository does not exist or may require 'docker login': denied: requested access to the resource is denied\n\n    /nuclio/pkg/cmdrunner/shellrunner.go:96\nFailed to build\n    /nuclio/pkg/dockerclient/shell.go:117\nFailed to build docker image\n    .../pkg/containerimagebuilderpusher/docker.go:54\nFailed to build processor image\n    /nuclio/pkg/processor/build/builder.go:263\nFailed to build processor image"}

Error - exit status 1
    /nuclio/pkg/cmdrunner/shellrunner.go:96

Call stack:
stdout:
Sending build context to Docker daemon  50.08MB
Step 1/12 : FROM cvat.openvino.dextr.base

stderr:
DEPRECATED: The legacy builder is deprecated and will be removed in a future release.
            Install the buildx component to build images with BuildKit:
            https://docs.docker.com/go/buildx/

pull access denied for cvat.openvino.dextr.base, repository does not exist or may require 'docker login': denied: requested access to the resource is denied

    /nuclio/pkg/cmdrunner/shellrunner.go:96
Failed to build
    /nuclio/pkg/dockerclient/shell.go:117
Failed to build docker image
    .../pkg/containerimagebuilderpusher/docker.go:54
Failed to build processor image
    /nuclio/pkg/processor/build/builder.go:263
Failed to deploy function
    ...//nuclio/pkg/platform/abstract/platform.go:197
  NAMESPACE |               NAME               | PROJECT |  STATE   | REPLICAS | NODE PORT
  nuclio    | openvino-dextr                   | cvat    | error    | 1/1      |
  nuclio    | openvino-omz-face-detection-0205 | cvat    | building | 1/1      |
  nuclio    | pth-facebookresearch-sam-vit-h   | cvat    | ready    | 1/1      |     32769

Error - exec: "/opt/nuclio/common/openvino/python3": stat /opt/nuclio/common/openvino/python3: no such file or directory     ...//nuclio/pkg/processor/runtime/rpc/abstract.go:239

I don't understand how that could be possible. I'm pretty sure I removed all references to /opt/nuclio/common/openvino/python3 from the repo. Maybe an earlier step failed?

When using ADD with more than one source file, the destination must be a directory and end with a /

This must be a slight difference between BuildKit and the old Docker builder. I made a change to explicitly enable BuildKit - could you try again? I also made the script stop after the first error.

I don't understand how that could be possible. I'm pretty sure I removed all references to /opt/nuclio/common/openvino/python3 from the repo. Maybe an earlier step failed?

Maybe it is related with #5603. I am trying to build on WSL.

Maybe it is related with https://github.com/opencv/cvat/issues/5603.

I doubt it.
The only reason Nuclio would be trying to execute /opt/nuclio/common/openvino/python3 is the NUCLIO_PYTHON_EXE_PATH variable, and... I removed it from everywhere. So there should really be no way for this string to even occur.

I am trying to build on WSL.

I've been developing on WSL this entire time. 🤷♂

[ ] Probably need to add installing docker-buildx-plugin to user guide.

Something wrong with DEXTR masks. It should not work this way:

Now all the changed functions (except the Deep extreme cut issue above) work for me.

Probably need to add installing docker-buildx-plugin to user guide.

Why?

Something wrong with DEXTR masks. It should not work this way:

Could you upload the original image, so I could test it myself?

Something wrong with DEXTR masks.

I think I figured it out - there were some postprocessing layers baked into the old IRs. I now added those to export.py - could you try again?

I think I figured it out - there were some postprocessing layers baked into the old IRs. I now added those to export.py - could you try again?

Now it works.

Why?

Because it didn't work without explicit installing for me.

Hi, @SpecLad does this PR mean, that I could run SAM via cvat helm chart? Or that is not possible right now? I tried to check the docs and do not see any info about how to do that.

@Keramblock You cannot deploy serverless functions themselves using the Helm chart, but you can deploy Nuclio with the chart (set the nuclio.enabled value to true), and then deploy SAM to that Nuclio instance using nuctl.

Oh, ok, thanks a lot for the clarification!


Updates to /examples

This PR corrects the /examples directory. Changes include:

[x] Remove /examples/indexer/java/java-test
[x] Rename /examples/indexer/go files as camelCase.js
[ ] Replace /examples/indexer/indexer.zip

[ ] Include functioning /examples/indexer/java/ project files

Incorrect repo. Closing.


PPS Dreambooth v2 doesn't work

I'm using Papserspace, and  the Dreambooth step has been stuck on "Training the text encoder..." for 10 minutes now.
It doesn't even begin the training process.
I've tried it twice on 2 different machines with different GPUs.
Please see if you can fix it.
V1 works fine though. Can I use the 768 model on version 1 without any issue?

UPDATE! Eventually it started the training process.
But after the training was done and I tried the model, I am now getting this error message: Error: Could not load the stable-diffusion model! Reason: Cannot copy out of meta tensor; no data!

choose a stronger machine

I mean I did pick a machine with 48GB Vram. So I don't see how that can be the issue.
Also, like I said, v1 works just fine.


ValidationRule javadocs

Description
Documentation for @ValidationRule and @RuleSet
Overview of Changes

Javadoc update

Type of Change

[ ] New feature
[ ] Bug fix
[ ] Refactor/Technical Debt
[X] Documentation Update
[ ] Test case change
[ ] This change requires a documentation update
[ ] Other

Test Details
N/A
Additional Notes
N/A

You did it @lopezton-anthem!
Thank you for signing the Anthem Opensource Software CLA.


Update GridSensor3D.gd - Fix non-uniform grid sizes

Fix for non-uniform grid sizes.

Ignore the accidental request removal. This is indeed ready for review and I have been using it when experimenting with the grid sensor, it should work.


Breaks almost everything "Pay with PayPal" related

New Issue Checklist (tick off like this: [x])

[x] I checked existing issues (open and closed) for possible duplicates
[x] I can reproduce the problem on the latest stable version (not nightly!)
[ ] If you have a problem a specific website: adding the domain to your blacklist does not solve the problem
[ ] The issue is not about adding entries to the default blacklist

What is your browser?

[x] Firefox
[ ] Chrome

What is your operating system?

[ ] Linux
[ ] Mac
[x] Windows

Description (please include examples/screenshots where applicable)
Title says it all - would be nice to have pay with PayPal popup functionality while also having this addon installed.

I don't use Paypal so I can't tell, but you probably need to add paypal to the blacklist or use whitelist mode.

I am using paypal.com/ in blacklist and never have a problem with paypal.
Cheers


What is the cron expression should be?

I am a newer that learn the cron expression. I want to use cron expression to achieve following requirements schedule. But i don't know how to write cron expression.
For example, i want to execute every 5 minutes during 16:20 pm and 18:40 pm every day. But it seems the one cron expression can't complete it. Any one has an good idea?

No, you would need several expressions.

I have actually rewritten the entire function to remove trailing conditions.


s2u-46: Fixed sorting bug on main screen

Could you fix the A11Y issue reported?

Feel free to mark it as ready for review once completed!


DOC-5659: Excluded the sidebar from search engine indexing

Fixes DOC-5659 and the following:

Tidies up the default and page layouts
Moves certain elements to include_cached for better performance

This change is


Move logging logic out of config file

Fixes #89 and as a by product, gets the per disc logging (the purpose of the logic) working correctly! It accomplishes this by moving the logic to the identify.sh script. Whenever this calls another script, it passes the LOG variable to that script. I have edited each of these scripts to use this parameter, so all of a discs activity will be logged to its own file.
Adds a new variable to config.sample as well, LOG_SINGLE_FILE, which defaults to false. With false, you get per disc logging and empty disc goes to empty.log
If you set it to true, it will log everything to ARM.log, much like the behavior before with everything going to empty.log
I thought some people might like the option to keep an eye on a single log file vs individual and this was an easy way to offer that choice.
Lots of changes for this to several files, I have double checked as well as tested out all the different disc types and it is working in my tests. Please test as well!

This looks really great.  I love the idea of getting the logic out of the config!

Has anyone else had a chance to test this PR yet? It has been working well for me but would love any feedback you may have.

Thanks!

My pleasure, love this project!
On Sat, Oct 28, 2017 at 1:55 PM, Benjamin Bryan<EMAIL_ADDRESS>wrote:

Thanks!
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/ahnooie/automatic-ripping-machine/pull/98#issuecomment-340208697,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AQVLdaBBfZThhrIdFHrKSYAi6ndry0nyks5sw2qtgaJpZM4P5ySW
.


Decouple test runtime from test core

Purpose
$Subject
Check List

[ ] Read the Contributing Guide
[ ] Updated Change Log
[ ] Checked Tooling Support (#)
[ ] Added necessary tests

[ ] Unit Tests
[ ] Spec Conformance Tests
[ ] Integration Tests
[ ] Ballerina By Example Tests

[ ] Increased Test Coverage
[ ] Added necessary documentation

[ ] API documentation
[ ] Module documentation in Module.md files
[ ] Ballerina By Examples

Codecov Report

Merging #21150 into master will increase coverage by 0.05%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master   #21150      +/-   ##
==========================================
+ Coverage   14.59%   14.64%   +0.05%
==========================================
  Files          51       51
  Lines        1398     1400       +2
  Branches      214      214
==========================================
+ Hits          204      205       +1
- Misses       1178     1179       +1
  Partials       16       16

Impacted Files
Coverage Δ

tool-plugins/vscode/src/core/extension.ts
28.03% <0%> (-0.14%)
:arrow_down:

tool-plugins/vscode/src/core/preferences.ts
100% <0%> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9c252a7...1d7fd6c. Read the comment docs.


Staking Payout Creates Controller

Fixes https://github.com/paritytech/substrate/issues/6415
This is a small update to the staking pallet that allows a staking payout to create a controller account. Note that this only works if the reward is greater than the existential deposit. If it is not, then the payout will be rounded down to zero and the account will not be created.

In practice, how much would someone need to be staked in order for this to not trash their rewards? Also, my understanding of the rewards system that nobody can hold off on claiming their rewards until they exceed the existential deposit, because anyone can trigger the semi-lazy payout of all nominators per (validator, era) pair.

If you want this, then maybe avoid the footgun by permitting staked accounts to own one account below the existential deposit.  I'd expect doing requires more logic than it's worth though.

The existential deposit on Polkadot is 0.01 DOT. According to polkadot apps, currently it will require > 18 DOT to have profit more than 0.01 DOT per era.


Feedback and idea for improvment

Thanks a lot for all of your amazing work!
This is by far the better and advanced GUI I tried.
I played a little bit with the GUI. I have some feedback:
Menu DNS:

When I add or Remove a Nameserver the GUI don't refresh automatically (I need to refresh the page I'm on firefox.)
For magic DNS it tells: Devices are accessible at [device].[user].<your-tailnet>: it's not always true if use_username_in_magic_dns: false then devices will be accessible with  [device].<your-tailnet>

Other:

I didn't find the tag of the version on the GUI or on docker-compose logs. For now I'm in 0.2.4, it will be useful to have this info (and will help debug people having issue).
From one of others headscale GUI, there was a Routes page: https://github.com/iFargle/headscale-webui/blob/main/screenshots/routes.png
This page was so useful. Are you planning to add such a view to see all distributed routes?

I was planning on adding a routes page at one point. The rest of these will be fixed in 0.3.0.


We cannot put username/password in code

Description:
It is not acceptable to have docker username/password in any program code.
Can we not use the local docker instance to push on our behalf? Most users have already logged into docker locally and this can be a prerequisite for the build. Alternatively, there could be some credential file stored locally or some secrets managed by ballerina.
Suggested Labels:
security
Suggested Assignees:
Affected Product Version:
All
OS, DB, other environment details and versions:
All
Steps to reproduce:
Related Issues:

Hi Paul,
we are thinking two option here. Already created issue for one option in docker support [1].  Second option is to put variable name for username and password, which can be resolved from env variable or toml file.
[1] https://github.com/ballerinax/docker/issues/9

Now the username and password(or any attribute) can be resolved using environment variables using $env{} syntaxes.
@kubernetes:Deployment{
    enableLiveness:"enable",
    push:true,
    image:"index.docker.io/$env{DOCKER_USERNAME}/gce-sample:1.0",
    username:"$env{DOCKER_USERNAME}",
    password:"$env{DOCKER_PASSWORD}"
}


Allowing for templated urls in iFrame

This can allow for passing {{ width }} and {{ height }} as dynamic
attributes in the iFrame's URL.
The new method Slice.render_template method could do more eventually
exposing more variables to be used in dynamic strings.

nice! LGTM.
will this be documented anywhere? another cool feature peeps should def know about!

I'm not sure where to document it just yet, it'd be nice to have some sort auto docs for the js api eventually. I know how to set that up in Python but not in JS yet.

Coverage remained the same at 81.627% when pulling faa14deb121a068e36a8138aa3d24bddcdd4b864 on mistercrunch:dynamic_mustaches into 6c333d50108f6ccb0c3997e9f687cfceb3e85552 on airbnb:master.

@williaster I added a note in the URL tooltip


Update comment on GRPC_ARG_TSI_MAX_FRAME_SIZE

This comment was added before frame size negotiation.
With frame size negotiation we can have a larger value for max frame size. Negotiation takes into consideration old binaries and non-zerocopy protectors like gRPC Go, and makes sure to agree on a frame size value that is acceptable at both endpoints.

Does this imply that ALTS has a frame size negotiation, and is it dynamic or just at handshake time?

Yes ALTS has frame size negotiation. It is only at handshake time.

And just to check, are there any other TSI supported transports which do not support frame size negotiation?

It is supported by LOAS2 (flag controlled) and ALTS for zero copy frame protectors (which as I understand is the default)
https://source.corp.google.com/piper///depot/google3/security/loas/l2/internal/transport_security/loas2_transport_security_common.cc;rcl=462721822;l=459
https://github.com/grpc/grpc/blob/ba8af0157ba9b39b6b584a8b5de3bd38a2c43588/src/core/tsi/alts/handshaker/alts_tsi_handshaker.cc#L175
The other transports like S2A and SSL do not have zero copy protectors, so they do not implement frame size negotiation


[WIP] Fix Nightly build

Motivation and context
Currently our nightly build is red (see here). This PR resolved this problem.
How has this been tested?

Checklist

[ ] I submit my changes into the develop branch
[ ] I have added a description of my changes into CHANGELOG file
[ ] I have updated the documentation accordingly
[ ] I have added tests to cover my changes
[ ] I have linked related issues (read github docs)
[ ] I have increased versions of npm packages if it is necessary (cvat-canvas,
cvat-core, cvat-data and cvat-ui)

License

[ ] I submit my code changes under the same MIT License that covers the project.
Feel free to contact the maintainers if that's a concern.

Now CI nightly is green: https://github.com/opencv/cvat/actions/runs/3044762850


[Feature] Support dependsOn in Component deployment DAG

Such as:

Or
apiVersion: core.oam.dev/v1beta1
kind: Application
metadata:
  name: wordpress-with-mysql
  namespace: default
spec:
  components:
    - name: mysql
      type: helm
      properties:
        repoType: helm
        url: https://charts.bitnami.com/bitnami
        chart: mysql
        version: "8.8.2"
    - name: wordpress
      type: helm
      inputs:
        # dependsOn the mysql component to be ready
        - dependsOn: mysql
      properties:
        repoType: helm
        url: https://charts.bitnami.com/bitnami
        chart: wordpress
        version: "12.0.3"

希望能够在Application里直观的看到到底哪些component存在依赖关系. 因为依赖关系是个比较通用的能力，执行过程都是DAG工作流，就看是否可以更友好一点

DependsOn and Inputs must be in the same configuration, otherwise, there will be conflict。
Like this?  Compatible with inputs as first-level parameters.
dependsOn:
 - name: mysql
    inputs:
    - from: connInfo
       parameterKey: connInfo
 - name: redis

The dependsOn should be a separate field defined like below:
components:
  - name: mysql
    type: helm
    properties:
      repoType: helm
      url: https://charts.bitnami.com/bitnami
      chart: mysql
      version: "8.8.2"
  - name: wordpress
    type: helm
    # dependsOn the mysql component to be ready
    dependsOn:
      - mysql
    properties:
      repoType: helm
      url: https://charts.bitnami.com/bitnami
      chart: wordpress
      version: "12.0.3"

How dependsOn works:

dependsOn is a syntax sugar. It will be translated to an input and an output

components:
  - name: mysql
    type: helm
    outputs:
    - name: mysql-startup-probe # translated output
      valueFrom: output.startupProbe.ready
  - name: wordpress
    type: helm
    inputs:
    - name: mysql-startup-probe  # translated input
      parameterKey: "" # empty, not filling any value

dependsOn will assume the Component it relies on, in this case the mysql, have startupProbe key in healthPolicy:
apiVersion: core.oam.dev/v1beta1
kind: ComponentDefinition
spec:
  status:
    healthPolicy: |
      startupProbe: op.#StartupProbe & {
        checkRule: (context.output.status.readyReplicas == context.output.status.replicas)
        failureThreshold: 30
        periodSeconds: 10
      }

Once the mysql component is ready, the startupProbe.ready key will set to true.

Why not reuse isHealth?
startupProbe is different from isHealth that it is only executed once, and indicates that the component is ready.
We need to differentiate them because startup and health checking are two separate operational stages.

So it is better to have a separate configuration to define the wait time, and check rule, etc.

Why not just make it as:
   ...
    - name: wordpress
      type: helm
      inputs:
        # dependsOn the mysql component to be ready
        - dependsOn: mysql
      properties:
       ...

It will be confuse if we allow dependsOn as a new field along with inputs.

I think it would be more clear to separate dependsOn and inputs, dependsOn is not a field inside of inputs though the inner implementation is a syntax sugar of inputs.

/assign

Conclusion from offline discussion:

We will add the dependsOn syntax sugar:
  - name: wordpress
    type: helm
    # dependsOn the mysql component to be ready
    dependsOn:
      - mysql

The working details are explained in my above proposal

When dependsOn and inputs co-exist, the dependsOn have higher priority to be ensured first before any data flow in inputs happen.
This can be implemented by prepending the translated input/output to the array as first element.

We will add the startupProbe action per my above proposal


Restricting Dataset Export for Specific User Groups

[Y] Read/searched the docs
[Y] Searched past issues

Currently, all users have access to the "Export as a dataset" function for all tasks. However, we need to restrict this functionality to only certain user groups. This way, we can ensure that only authorized users have the ability to download datasets, while allowing all users to view tasks and perform annotations. Any guidance on how to achieve this would be highly appreciated. Thank you!

Currently, all users have access to the "Export as a dataset" function for all tasks

This is not true. Export as a dataset for a task is available only for task owner, task assingee, project owner and project assignee.
If the task is in an organization, also for organization owner and maintainer. And for any user who has admin role.

In general if you want to modify any permissions, you need to modify rego rules in this directory cvat/apps/iam/rules and then rebuild CVAT.
This article also can be useful, but outdated a little bit.

FIRSTLT:
change two rego file:
(apps/iam/rules/jobs.rego
apps/iam/rules/`tasks.rego)
@@ -156,7 +156,6 @@ allow {

 allow {
     { utils.VIEW,
-      utils.EXPORT_DATASET, utils.EXPORT_ANNOTATIONS,
       utils.VIEW_ANNOTATIONS, utils.VIEW_DATA, utils.VIEW_METADATA
     }[input.scope]
     utils.is_sandbox
@@ -165,7 +164,6 @@ allow {

 allow {
     { utils.CREATE, utils.DELETE, utils.VIEW,
-      utils.EXPORT_DATASET, utils.EXPORT_ANNOTATIONS,
       utils.VIEW_ANNOTATIONS, utils.VIEW_DATA, utils.VIEW_METADATA
     }[input.scope]
     input.auth.organization.id == input.resource.organization.id
@@ -175,7 +173,6 @@ allow {

 allow {
     { utils.VIEW,
-      utils.EXPORT_DATASET, utils.EXPORT_ANNOTATIONS,
       utils.VIEW_ANNOTATIONS, utils.VIEW_DATA, utils.VIEW_METADATA
     }[input.scope]
     input.auth.organization.id == input.resource.organization.id
@@ -183,6 +180,12 @@ allow {
     is_job_staff
 }

+allow {
+    { utils.EXPORT_DATASET
+    }[input.scope]
+    input.auth.user.privilege == utils.admin
+}
+

allow {
    { utils.EXPORT_DATASET
    }[input.scope]
    input.auth.user.privilege == utils.admin
}

SECONDLY
docker compose -f docker-compose.yml -f docker-compose.dev.yml build
LASTLY
docker compose up -d

Hello @lxy5513 ,
Could you help me how you add "return Response(status=status.HTTP_400_BAD_REQUEST)" to the CVAT codes?
I changed cvat/cvat/apps/engine/rules/jobs.rego but still worker users can export tasks.
Thank you in advance.


add photos along a trip route

We need to be able to place and display the photos along a route. Ideally OS would allow the user to do that manually and via gps. I'll add a separate ticket in carless, but will apply to OS as well.
similar to how strava does it seems to be pretty standard.

this is done, I followed the same design as for the other features in the map. clicking on a photo marker will make that photo appear in the carousel.


OrientDB3.0.0m1 Graph UI rendering issue

OrientDB Version: 3.0.0m1 docker
Java Version: 1.8.0_121
OS: ubuntu
Expected behavior
I installed 3.0.0m1 via docker images from hub.docker.io, then start docker, from the studio front page, import existing GamesOfThrones and logged in.
In the Graph tab page, one simple search like "select * from Event", the graph ui is not rendered.
From the browser debug console, the js error throws out.
Is the cloud databases are out of date and incompatible with 3.0.0? Or something else?

main.js:165528 TypeError: Cannot read property 'data' of undefined
at main.js:191560
at processQueue (main.js:168024)
at main.js:168040
at Scope.$eval (main.js:169331)
at Scope.$digest (main.js:169145)
at Scope.$apply (main.js:169439)
at done (main.js:163420)
at completeRequest (main.js:163629)
at XMLHttpRequest.requestLoaded (main.js:163557)
at XMLHttpRequest.wrapFn [as _onload] (main.js:247150)
at ZoneDelegate.invokeTask (main.js:246561)
at Object.onInvokeTask (main.js:50584)
at ZoneDelegate.invokeTask (main.js:246560)
at Zone.runTask (main.js:246450)
at XMLHttpRequest.ZoneTask.invoke (main.js:246631)
TypeError: Cannot read property '0' of undefined
at Object.push (main.js:197196)
at main.js:191571
at processQueue (main.js:168024)
at main.js:168040
at Scope.$eval (main.js:169331)
at Scope.$digest (main.js:169145)
at Scope.$apply (main.js:169439)
at done (main.js:163420)
at completeRequest (main.js:163629)
at XMLHttpRequest.requestLoaded (main.js:163557)
at XMLHttpRequest.wrapFn [as _onload] (main.js:247150)
at ZoneDelegate.invokeTask (main.js:246561)
at Object.onInvokeTask (main.js:50584)
at ZoneDelegate.invokeTask (main.js:246560)
at Zone.runTask (main.js:246450)

Hi @NicholasQu
thanks for your feedback. Let me check

This might be the same problem (same error): https://github.com/orientechnologies/orientdb/issues/7297

I can't check the similarity with @karolmie1 mentioned, if they are same, someone can help to close it.


No license specified

There is no license specified for this package - I can add in another pull request for adding a BSD license.
Before I do so, who do I add the copyright under?

I don't know. I guess the best thing to do would be to figure out where it came from and look at that (I think was extracted from a large repository at some point). @thomas-moulard has the oldest commits.

The package.xml file was stating BSD in the first commit of this repo:
https://github.com/ros/urdf_parser_py/commit/ca7453d9fb86c2cc4f16fd83ecd9945a517fd386
Note that this is a fork of urdf_python by David!! so you may want to include him in the license file if you put my name. On my side, I don't have strong preference so on my side I allow you to add/edit/remove licensing information to align this package with what is done in ROS usually. You can also add my name to the license file if needed.

We've switched over to the indigo-devel branch, which lists a BSD license in the package.xml, so I'm going to close this out.


CF CLI reporting serialization error on login with --sso

Please fill out the issue checklist below and provide ALL the requested information.

[X ] I reviewed open and closed github issues that may be related to my problem.
[X ] I tried updating to the latest version of the CF CLI to see if it fixed my problem.
[X ] I attempted to run the command with CF_TRACE=1 to help debug the issue.
[X ] I am reporting a bug that others will be able to reproduce.
[NA ] If this is an issue for the v7 beta release, I've read through the official docs and the release notes.

Describe the bug and the command you saw an issue with
I'm having trouble logging in to an on-prem PCF environment using SSO. I've tested using an identical version of the CLI as a colleague who is not having the trouble, but using the slightly older version (6.48) didn't change any behavior.
The core issue is that I'm receiving an error, json: cannot unmarshal string into Go struct field InfoLinks.links.uaa of type ccv3.APILink in response to my login --sso request.
> cf login -a <https-url> --sso
What happened
First, I got a (spurious?) SSL response to my login attempt:
> cf login -a <https-url> --sso
> Invalid SSL Cert for <https-url> TIP: Use 'cf login --skip-ssl-validation' to continue with an insecure API endpoint
I say spurious because a colleague on a similar Mac is able to execute this command without the SSL error.
Next, I tried to login with the SSL-skip flag:
> cf login -a <https://login...> --sso --skip-ssl-validation
> json: cannot unmarshal string into Go struct field InfoLinks.links.uaa of type ccv3.APILink
Running the same command with CF_TRACE=true yields this output:
`RESPONSE: [2020-07-22T08:45:49-05:00]
HTTP/1.1 200 OK
Cache-Control: no-store
Content-Language: en-US
Content-Length: 764
Content-Type: application/json;charset=UTF-8
Date: Wed, 22 Jul 2020 13:45:48 GMT
Set-Cookie: X-Uaa-Csrf=68OdX8qO4cg6BY9zoabl1K; Max-Age=86400; Expires=Thu, 23-Jul-2020 13:45:49 GMT; Path=/; Secure; HttpOnly
Strict-Transport-Security: max-age=31536000 ; includeSubDomains
X-Content-Type-Options: nosniff
X-Frame-Options: DENY
X-Vcap-Request-Id: 6a443ed4-7666-4472-5c4a-2a9aac3d5321
X-Xss-Protection: 1; mode=block
{
"app": {
"version": "73.4.24"
},
"commit_id": "a91c1a7",
"entityID": "",
"idpDefinitions": {
"https://saml...": "https://login...?returnIDParam=idp&entityID=http://login...&idp=https://saml...&isPassive=true"
},
"links": {
"login": "https://login...",
"uaa": "https://uaa..."
},
"prompts": {
"passcode": [
"password",
"Temporary Authentication Code ( Get one at https://login.../passcode )"
],
"password": "[PRIVATE DATA HIDDEN]",
"username": [
"text",
"Email"
]
},
"showLoginLinks": true,
"timestamp": "2020-05-19T19:05:36+0000",
"zone_name": "uaa"
}`
Expected behavior
The first login command (without the SSL override) should work.
To Reproduce
See above
Provide more context

platform and shell details: Mac OS X 10.15.5,  iTerm2 using ZSH
cf version 6.51.0+2acd15650.2020-04-07

@jeblackburn - are you able to reproduce this issue still with the latest v6 CLI (6.53.0) and/or v7 CLI (7.1.0)?

I'm having same issue --v cf version 7.2.0+be4a5ce2b.2020-12-10, Cloud Foundry command line tool

Thank you for your submission. If you still run into this issue with the latest CLI v8,  please open a new ticket.


NIFI-11125: Adding unset property handling to AbstractDBCPConnectionPool

Summary
NIFI-11125
Tracking
Please complete the following tracking steps prior to pull request creation.
Issue Tracking

[x] Apache NiFi Jira issue created

Pull Request Tracking

[x] Pull Request title starts with Apache NiFi Jira issue number, such as NIFI-00000
[x] Pull Request commit message starts with Apache NiFi Jira issue number, as such NIFI-00000

Pull Request Formatting

[x] Pull Request based on current revision of the main branch
[x] Pull Request refers to a feature branch with one commit containing changes

Verification
Please indicate the verification steps performed prior to pull request creation.
Build

[ ] Build completed using mvn clean install -P contrib-check

[ ] JDK 8
[ ] JDK 11
[ ] JDK 17

Licensing

[ ] New dependencies are compatible with the Apache License 2.0 according to the License Policy
[ ] New dependencies are documented in applicable LICENSE and NOTICE files

Documentation

[ ] Documentation formatting appears as expected in rendered files

Based on offline discussion with @Lehel44, closing this pull request in favor of evaluation a different approach to code reuse for other Connection Pool services.


Adding Kings School

Adding the kings.edu.au domain to schools in Australia

@dlhegarty Pull request merged. Thank you!


Investigate improvements that could speed MarkDuplicates

Right now, mark_duplicates_and_sort takes about as much time as aligning reads on large WGS bams. Samblaster can be used inline, which should save a lot of I/O and improve speed:
bwa mem <idxbase> samp.r1.fq samp.r2.fq | samblaster | samtools view -Sb - > samp.out.bam

Sorting could also potentially be added in that pipe, as in the speedseq pipeline. Back of the envelope, I'm guessing we could shave 4-8 hours off of WGS alignments with this approach.

Samblaster is already called in-line to add mate tags. However, it also is explicitly using the -a option to accept existing dup markings. I'm pretty sure this was explicitly configured this way to bypass samblaster duplicate marking and later use Picard MarkDuplicates. Also, if I recall correctly, there were reasons to use MarkDuplicates to maintain "functional equivalence" with other alignment workflows, ex. CCDG.
If we do want to run samblaster for duplicate marking, the changes should be fairly straightforward. As an example drop the -a from:
https://github.com/genome/docker-alignment_helper-cwl/blob/8c816f3cfdaa68091a83028611f712e50de5185c/alignment_helper.sh#L6
Then remove the Picard MarkDuplicates step from the alignment subworkflow.

Yeah, I remember that old discussion, and I think that for somatic calling, at least, we don't care about being 100% aligned with the CCDG pipeline (they're not even using these workflows, right?)

Never mind... isn't the real issue merging? Speedseq as an example required all of the inputs for a sample and aligned in out BWA process/stream. Most of MGI's alignment workflows assume we will align each input separately in a scatter. Marking duplicates requires all reads in order to mark the correct duplicates.

Ugh.  You're right.  Even though a lot of data comes in as single sets of fastqs these days, we can't rely on that. I also don't think merging the fastqs up front would be worth the hit we would take from that merge and from losing the ability to scatter the alignments.  Okay, closing this one up for now.

Poking around, there's this option that's worth investigating
https://gatk.broadinstitute.org/hc/en-us/articles/360037224932-MarkDuplicatesSpark

MarkDuplicatesSpark is optimized to run locally on a single machine by leveraging core parallelism that MarkDuplicates and SortSam cannot. It will typically run faster than MarkDuplicates and SortSam by a factor of 15% over the same data at 2 cores and will scale linearly to upwards of 16 cores. This means MarkDuplicatesSpark, even without access to a Spark cluster, is faster than MarkDuplicates.

In addition to spark/multi-threading improvements, we should re-evaluate the order of operations in our workflow. I don't believe older versions of MarkDuplicates would take multiple input files, so effectively MarkDuplicates can be a merge step as well rather than having a separate independent merge. As an example, check out the Broad Best Practice WDL:
https://github.com/gatk-workflows/gatk4-data-processing/blob/master/processing-for-variant-discovery-gatk4.wdl

I tried to run the MarkDuplicatesSpark tool in the 'broadinstitute/gatk:<IP_ADDRESS>' container, but get this completely unhelpful error:
java.io.IOException: failure to login: javax.security.auth.login.LoginException: java.lang.NullPointerException: invalid null input: name
        at com.sun.security.auth.UnixPrincipal.<init>(UnixPrincipal.java:71)
        at com.sun.security.auth.module.UnixLoginModule.login(UnixLoginModule.java:133)

The documentation for this is rather lacking:
https://gatk.broadinstitute.org/hc/en-us/articles/360035889871--How-to-Run-Spark-enabled-GATK-tools-on-a-local-multi-core-machine
After some searching in non-gatk contexts, I suspect that this has something to do with not having the expected access permissions inside the docker container. Specifically, this stackoverflow question:
https://stackoverflow.com/questions/45198252/apache-spark-standalone-for-anonymous-uid-without-user-name
Spark wants to be able to look up its UID in passwd. This integration kink can be resolved using nss_wrapper
. . .

As well as this page:
https://docs.cloudera.com/runtime/7.2.1/yarn-troubleshooting/topics/yarn-troubleshooting-docker.html
Problem statement
    Container fails shortly after start with the following exception:

    Exception in thread "main" org.apache.hadoop.security.KerberosAuthException: failure to login: javax.security.auth.login.LoginException: java.lang.NullPointerException: invalid null input: name
    	At com.sun.security.auth.UnixPrincipal.<init>(UnixPrincipal.java:71)
    	at com.sun.security.auth.module.UnixLoginModule.login(UnixLoginModule.java:133)
    	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl. java:62)
    	at sun.reflect.DelegatingMethodAccessorImpl.invoke( DelegatingMethodAccessorImpl.java:43)

Root cause
    The Hadoop UserGroupInformation class does not have access to the user permissions in the host system.
Resolution
    Mount the /etc/passwd to the image. More configuration issues can be found in upstream Hadoop 3.2 documentation: Launching Applications Using Docker Containers upstream documentation.

This question also suggests some fixes, all of which require low-level access that we don't have, due to our unusual cluster architecture: https://stackoverflow.com/questions/41864985/hadoop-ioexception-failure-to-login
At this point, I'm unclear how to proceed next here

Out of curiosity, where are you trying to run this?  The newer cluster does supply an /etc/passwd, but on the older one I suspect you'd need to do something along the lines of this: https://confluence.ris.wustl.edu/pages/viewpage.action?pageId=27593241

I've circled back to this, and it does indeed work on compute1, which is encouraging. I have a test running there, and the next step will be to see if we can fork the gatk container to add libnss-sss (or the equivalent) so that it will work on compute0 as well. If not, this becomes a non-backwards compatible change that breaks on compute0

The current state of this is that Mark Duplicates isn't speedy, but it is not as time-consuming as previously though. The batch of jobs we were benchmarking on were during a time of high cluster usage, and time spent waiting to land a job was included in the cromwell chart.  The other side of this is that the Spark implementation does not seem to be substantially faster than the single core method on our cluster (despite using substantially more resources) seemingly due to excessive I/O.  I'm going to close this issue for now, and we can revisit at some later date if it becomes a bottleneck once again.


Export Exasol tables to Azure and GCP storages

Similarly to S3 export functionality,
as a user of cloud-storage-etl-udfs I want to export exasol tables into Azure and GCP object storages.

Fixed in #20


bazel compile error: enumeration value ‘CUDNN_POINTWISE_RECIPROCAL’ not handled in switch

Click to expand!
Issue Type
Build/Install
Have you reproduced the bug with TF nightly?
No
Source
source
Tensorflow Version
v2.12.0
Custom Code
Yes
OS Platform and Distribution
Linux Debian Bullseye
Mobile device
No response
Python version
3.9
Bazel version
5.3.0
GCC/Compiler version
gcc (Debian 10.2.1-6) 10.2.1 20210110
CUDA/cuDNN version
cuda_11.6.r11.6/compiler.31057947_0
GPU model and memory
RTX A4000
Current Behaviour?
bazel build --config=cuda --config=mkl //tensorflow/tools/pip_package:build_pip_package
failed displaying following error messages
ERROR: /mnt/data/kentwork/src/AI/tensorflow/tensorflow/compiler/xla/stream_executor/cuda/BUILD:376:11: Compiling tensorflow/compiler/xla/stream_executor/cuda/cuda_dnn.cc failed: (Exit 1): crosstool_wrapper_driver_is_not_gcc failed: error executing command external/local_config_cuda/crosstool/clang/bin/crosstool_wrapper_driver_is_not_gcc -MD -MF bazel-out/k8-opt/bin/tensorflow/compiler/xla/stream_executor/cuda/_objs/cudnn_plugin/cuda_dnn.pic.d ... (remaining 141 arguments skipped)
In file included from bazel-out/k8-opt/bin/external/cudnn_frontend_archive/_virtual_includes/cudnn_frontend/third_party/cudnn_frontend/include/cudnn_frontend_Operation.h:37,
from bazel-out/k8-opt/bin/external/cudnn_frontend_archive/_virtual_includes/cudnn_frontend/third_party/cudnn_frontend/include/cudnn_frontend_OperationGraph.h:36,
from bazel-out/k8-opt/bin/external/cudnn_frontend_archive/_virtual_includes/cudnn_frontend/third_party/cudnn_frontend/include/cudnn_frontend_Heuristics.h:31,
from bazel-out/k8-opt/bin/external/cudnn_frontend_archive/_virtual_includes/cudnn_frontend/third_party/cudnn_frontend/include/cudnn_frontend.h:101,
from tensorflow/compiler/xla/stream_executor/cuda/cuda_dnn.cc:55:
bazel-out/k8-opt/bin/external/cudnn_frontend_archive/_virtual_includes/cudnn_frontend/third_party/cudnn_frontend/include/cudnn_frontend_PointWiseDesc.h: In member function ‘int64_t cudnn_frontend::PointWiseDesc_v8::getPortCount() const’:
bazel-out/k8-opt/bin/external/cudnn_frontend_archive/_virtual_includes/cudnn_frontend/third_party/cudnn_frontend/include/cudnn_frontend_PointWiseDesc.h:69:16: error: enumeration value ‘CUDNN_POINTWISE_RECIPROCAL’ not handled in switch [-Werror=switch]
69 |         switch (mode) {
|                ^
In file included from bazel-out/k8-opt/bin/external/cudnn_frontend_archive/_virtual_includes/cudnn_frontend/third_party/cudnn_frontend/include/cudnn_frontend_OperationGraph.h:36,
from bazel-out/k8-opt/bin/external/cudnn_frontend_archive/_virtual_includes/cudnn_frontend/third_party/cudnn_frontend/include/cudnn_frontend_Heuristics.h:31,
from bazel-out/k8-opt/bin/external/cudnn_frontend_archive/_virtual_includes/cudnn_frontend/third_party/cudnn_frontend/include/cudnn_frontend.h:101,
from tensorflow/compiler/xla/stream_executor/cuda/cuda_dnn.cc:55:
bazel-out/k8-opt/bin/external/cudnn_frontend_archive/_virtual_includes/cudnn_frontend/third_party/cudnn_frontend/include/cudnn_frontend_Operation.h: In member function ‘cudnn_frontend::Operation_v8&& cudnn_frontend::OperationBuilder_v8::build_pointwise_op()’:
bazel-out/k8-opt/bin/external/cudnn_frontend_archive/_virtual_includes/cudnn_frontend/third_party/cudnn_frontend/include/cudnn_frontend_Operation.h:354:16: error: enumeration value ‘CUDNN_POINTWISE_RECIPROCAL’ not handled in switch [-Werror=switch]
354 |         switch (m_operation.pointwise_mode) {
|                ^
cc1plus: some warnings being treated as errors
Target //tensorflow/tools/pip_package:build_pip_package failed to build
Use --verbose_failures to see the command lines of failed build steps.
Standalone code to reproduce the issue
bazel build --config=cuda --config=mkl //tensorflow/tools/pip_package:build_pip_package

Relevant log output
No response

It looks like you are encountering a compilation error when building TensorFlow with Bazel on your Debian Bullseye system using CUDA and MKL. The error is related to a missing switch case in the cudnn_frontend library.
One possible solution to this issue is to update the cudnn_frontend library to a newer version that has the missing switch case. You can try updating the cudnn_frontend by running the following command:
arduino
Copy code
bazel build --config=cuda --config=mkl --config=monolithic //tensorflow/tools/pip_package:build_pip_package --incompatible_remove_native_http_archive=false
This command will build TensorFlow with CUDA, MKL, and the latest version of the cudnn_frontend library.
If updating the cudnn_frontend library doesn't solve the issue, you can try disabling the --config=mkl option when building TensorFlow with Bazel. You can do this by running the following command:
arduino
Copy code
bazel build --config=cuda //tensorflow/tools/pip_package:build_pip_package
This command will build TensorFlow with only CUDA and without MKL. This might help you to identify whether the issue is related to the MKL library or not.
If neither of these solutions works, you might want to provide more information such as the complete build log or the full error message to help us better understand the issue.

$ bazel build --config=cuda --config=mkl --config=monolithic //tensorflow/tools/pip_package:build_pip_package --incompatible_remove_native_http_archive=false
ERROR: --incompatible_remove_native_http_archive=false :: Unrecognized option: --incompatible_remove_native_http_archive=false
gives Unrecognized option error
I also tried removing --config mkl, which I can not get any improvement.

https://github.com/tensorflow/tensorflow/issues/60362 FYI.
btw:
tensorflow V2.12.0 requires 8.6cuDNN and 11.8 CUDA

Hi @kentosho ,
Please look at the tested configurations here. For Tf2.12 version the tested configurations are GCC-9.3.1 , CUDA-11.8,cuDNN-8.6. Can you please cross check with the tested configurations.
Also I would like to know the sequence of steps followed for build.Have you installed cuDNN as per instructions ? I am interested to know the sequence of steps followed by you for GPU setup with CUDA,cuDNN toolkit installations etc. Official instructions as per documentation mentioned here.

compile suceeded for gcc-10.2.1 CUDA-11.8 cuDNN-8.9 with following patch
it might be a wired patch
diff --git a/tensorflow/workspace2.bzl b/tensorflow/workspace2.bzl
index 1261273bc92..0a9d755af0f 100644
--- a/tensorflow/workspace2.bzl
+++ b/tensorflow/workspace2.bzl
@@ -172,9 +172,9 @@ def _tf_repositories():
name = "cudnn_frontend_archive",
build_file = "//third_party:cudnn_frontend.BUILD",
patch_file = ["//third_party:cudnn_frontend_header_fix.patch"],

   sha256 = "3c7b842cd67989810955b220fa1116e7e2ed10660a8cfb632118146a64992c30",

   strip_prefix = "cudnn-frontend-0.7.3",

   urls = tf_mirror_urls("https://github.com/NVIDIA/cudnn-frontend/archive/refs/tags/v0.7.3.zip"),

   sha256 = "d8dba9e2607a0c256aa8eacb45b39986ab6f3f24a4d431d4397047a3cb0cd4fb",

   strip_prefix = "cudnn-frontend-0.9",

   urls = tf_mirror_urls("https://github.com/NVIDIA/cudnn-frontend/archive/refs/tags/v0.9.zip"),

)

Hi @kentosho ,
This might be due to the fact that TF tested cuDNN version is 8.6 whereas you are trying to use different version 8.9 and hence the change in dependencies might be worked.
I assume you haven't checked with cuDNN-8.6 which is a tested configuration. If you have tested this but still fails please let us know. have you done the patch suitable for cuDNN-8.9 directly please let us know.

I try to build with cuDNN 8.6 instead of cuDNN8.9 and sucssess to build with no error.　Also, when using CuDNN 8.9, I had experienced an abnormal termination of python when running model.fit on sensorflow 2.12.0, but this phenomenon was　fixed by using CuDNN 8.6.

It looks like you are encountering a compilation error when building TensorFlow with Bazel on your Debian Bullseye system using CUDA and MKL. The error is related to a missing switch case in the cudnn_frontend library.
One possible solution to this issue is to update the cudnn_frontend library to a newer version that has the missing switch case. You can try updating the cudnn_frontend by running the following command:
arduino Copy code bazel build --config=cuda --config=mkl --config=monolithic //tensorflow/tools/pip_package:build_pip_package --incompatible_remove_native_http_archive=false This command will build TensorFlow with CUDA, MKL, and the latest version of the cudnn_frontend library.
If updating the cudnn_frontend library doesn't solve the issue, you can try disabling the --config=mkl option when building TensorFlow with Bazel. You can do this by running the following command:
arduino Copy code bazel build --config=cuda //tensorflow/tools/pip_package:build_pip_package This command will build TensorFlow with only CUDA and without MKL. This might help you to identify whether the issue is related to the MKL library or not.
If neither of these solutions works, you might want to provide more information such as the complete build log or the full error message to help us better understand the issue.

Is this a chatgpt reply?


[TE] Update segment creation API

Update API call to the latest Pinot library.

this has already been done by Alex https://github.com/linkedin/pinot/pull/2159


example for camera_platform_interface

hello every one
i want to create a video recorder with limited time duration
i check Camera package default  dont has this option and it need to use camera_platform_interface
but with camera_platform_interface dont has any example source
Can anyone help me ?

The perform interface is not intended for direct use; it's an implementation detail of federation. If you need functionality that is not in the app-facing API, please file a feature request with details of the request and use case.
Closing as we do not plan to document this for direct use.


chore(mainnet): improved xcm config, support mainnet in xcm tests

Codecov Report
Attention: Patch coverage is 50.00000% with 1 line in your changes missing coverage. Please review.

Project coverage is 11.21%. Comparing base (d28f7bf) to head (abfbc22).

Files with missing lines
Patch %
Lines

node/src/chain_spec.rs
50.00%
1 Missing :warning:

@@                      Coverage Diff                       @@
##           al3mart/chore-registration     #247      +/-   ##
==============================================================
- Coverage                       11.24%   11.21%   -0.03%
==============================================================
  Files                              34       34
  Lines                            2580     2577       -3
  Branches                         2580     2577       -3
==============================================================
- Hits                              290      289       -1
+ Misses                           2287     2285       -2
  Partials                            3        3

Files with missing lines
Coverage Δ

runtime/mainnet/src/config/xcm.rs
0.00% <ø> (ø)

runtime/mainnet/src/lib.rs
23.95% <ø> (ø)

node/src/chain_spec.rs
11.29% <50.00%> (-0.22%)
:arrow_down:


Use "more entropy" option when using uniqid() function

As you can see in the source code of the PHP uniqid() function when you don't pass true as the second argument, PHP sleeps the application for 1 microsecond (usleep(1))
Symfony uses true almost everywhere, but this bundle still had some uniqid() calls without this option.

I was not aware of the usage of usleep in this case.

Thank you @javiereguiluz.


Use of undeclared type of SectionedViewDataSourceType on iPhone4s simulator

Hello
I have an issue building project with iPhone4s Simulator (9.0)
My project-swift language is 3.3.
- RxDataSources (1.0.4):
    - RxCocoa (~> 3.0)
    - RxSwift (~> 3.0)

I got the issue that use of undeclared type of SectionedViewDataSourceType.
That's also strange, I can build iPhone5s(9.0) Simulator.

Changing the RxDataSources (in Pods) optimization level to Speed [-O] fixes the issue.


Using "warm" with sst.aws.Astro breaks lambda handler

Hello! I have an sst ion stack that contains an Astro site, and I tried to enable "warm" for the site:
const astro = new sst.aws.Astro("AstroSite", {
  domain: astroDomain,
  path: "packages/astro-site",
  warm: 1
});

Without the "warm" property, the site deploys fine and is reachable through the astroDomain.
But when I change to warm: 1, I get this error when trying to hit the same url:

This results in a 502 Bad Gateway response from cloudfront that's in front of the astro function.
It also seems that the Runtime configuration of the lambda changes if I inspect in in the AWS console ui, Handler changes from packages/astro-site/dist/server/entry.handler to packages/astro-site/dist/server/server-index.handler.
My astro.config.mjs:
import { defineConfig } from "astro/config";
import tailwind from "@astrojs/tailwind";
import aws from "astro-sst";
import preact from "@astrojs/preact";

// https://astro.build/config
export default defineConfig({
  output: "server",
  adapter: aws(),
  integrations: [tailwind({ applyBaseStyles: false }), preact()]
});

Dependencies:

sst cli: 0.0.393
sst node module: 3.0.32
astro: 4.9.2
astro-sst: 2.42.0

I believe this will be fixed one #469 is merged.


Added local network permission on IOS

Why it is needed?
With the new Apple update 14.0, a new permission is requested: app name wants to search for devices on your local network and connect to them. This app will be able to discover and connect to devices on the networks you use.
Thank you :)

Duplicate https://github.com/zoontek/react-native-permissions/issues/509


Implement PWA Feature

Can you provide a little description on this one?

Can you provide a little description on this one?

@gitshashwat This might help you : https://medium.com/the-web-tub/creating-your-first-vue-js-pwa-project-22f7c552fb34


The [alt] button below featured images is no longer functional.

Issue Summary
After the latest Ghost updates, the button below featured images to put in alt text is no longer functional.
Steps to Reproduce
Open the latest version of Ghost.
Add a post.
Add a featured image to the post.
Click the [alt] button next to the caption to switch to entering alt text.
Nothing happens.  I expected to be able to enter alt text.
The problem occurs in both the original and new beta editor.
The problem seems localized to featured images.  Other images are not impacted.
The problem is present in Chrome and Edge both.  No error messages are present in the browser console.
Ghost Version
5.50-something
Node.js Version
Ghost Pro
How did you install Ghost?
Ghost Pro
Database type
MySQL 8
Browser & OS version
Chrome 114.0.5735.90 on Windows 10 AND Edge (recent version)
Relevant log / error output
No response
Code of Conduct

[X] I agree to be friendly and polite to people in this repository

This is now fixed in v5.50.3 🙂


Fix SLLZ compression

Description
I was assumming incorrectly that first flag in SLLZ v1 compression was always 0.
This closes #11

@SutandoTsukai181 could you test if this fixes the problem?

Just tested, and yes the problem is fixed. Resulting compressed par can be decompressed now.


Patches fail to apply on Fedora 25

Some patches fail to apply due to different line endings:
  patch -p1 --ignore-whitespace -i /home/tmds/repos/source-build/patches/nuget-client/0001-Revert-38949ce93081016167dfd8e37e4e897bedf47d65-DOS-.patch
  (Stripping trailing CRs from patch; use --binary to disable.)
  patching file build/common.project.props
  Hunk #1 FAILED at 14 (different line endings).
  1 out of 1 hunk FAILED -- saving rejects to file build/common.project.props.rej
  (Stripping trailing CRs from patch; use --binary to disable.)
  patching file src/NuGet.Core/NuGet.Build.Tasks.Pack/NuGet.Build.Tasks.Pack.csproj
  Hunk #1 FAILED at 5 (different line endings).
  Hunk #2 FAILED at 51 (different line endings).
  2 out of 2 hunks FAILED -- saving rejects to file src/NuGet.Core/NuGet.Build.Tasks.Pack/NuGet.Build.Tasks.Pack.csproj.rej

This is not failing in CI since it is using a different patch command
https://github.com/dotnet/source-build/blob/14d33dc2af2be062d842ff93c8410e34b6b89a93/targets/repository.proj#L45-L46
CI should use the same command as non-CI builds.
Can we use the git apply command always (since that is working)? Or should we use patch also in CI?
CC @crummel @omajid

As I understand it, in our build-for-distribution scenario, we are building from a source tarball and dont have a clone of the git repo. Since git is not generally needed, a previous maintainer thought it was better to not have a build dependency on git at all. He added the support for patch.
If patch vs git is causing issues, I think we should just take a build-time dependency on git (for git apply) and stop worrying about differences in how they apply patches. We have too many other things to worry about already :)

Fixed by https://github.com/dotnet/source-build/pull/135


Symlink loop

I'm using Vagrant for testing and it rejects to sync common scripts to the virtual machine (using rsync) because there is an infinite loop of symlinks: common/tests/failures/check contains common which is a symlink back to ../../.. which causes this loop:
common/tests/failures/check/common/tests/failures/check/common/tests/failures/check/common/tests/failures/check/common…

Is this something we need to have here?

This was introduced by https://github.com/sclorg/container-common-scripts/pull/60.

I barely remember why I did this, but it was to make the test suite work in this repo.
Why rsync complaints?  Do you really need to use rsync -L ?

You are right, I can reconfigure Vagrant to either ignore that folder or do not use the --copy-links default option: https://www.vagrantup.com/docs/synced-folders/rsync#options
But, and it might be just my feeling, symlink loops are weird and I'd rather avoid them, if possible. And according to the proposed change, it should be possible and it also tests a way of use that has not been tested before.