Credential issuer is required.

Appears something has changed here...
https://github.com/digitalbazaar/vc-js/blob/ba2927080ff6cc20441d3232b9f087d8f7807bd4/lib/CredentialIssuancePurpose.js#L59
I am passing:
{
  "@context": [
    "https://www.w3.org/2018/credentials/v1",
    "https://www.w3.org/2018/credentials/examples/v1"
  ],
  "id": "http://example.gov/credentials/3732",
  "type": [
    "VerifiableCredential",
    "UniversityDegreeCredential"
  ],
  "issuer": "did:web:vc.transmute.world",
  "issuanceDate": "2020-03-10T04:24:12.164Z",
  "credentialSubject": {
    "id": "did:example:ebfeb1f712ebc6f1c276e12ec21",
    "degree": {
      "type": "BachelorDegree",
      "name": "Bachelor of Science and Arts"
    }
  }
}

Confused why the issuer is not being picked up.... possible that its expected format has changed?

Also note this link is not correct: https://www.w3.org/2018/credentials/#issuer

Caused by not specifying proof purpose:
 purpose: new AssertionProofPurpose(),
in the verify function.

I'm slightly confused why we are continuing to create new proof purposes, without a formal review process... we have very limited support for the existing purposes... every new one we add, will be met with objection, and decrease interop with existing implementations.

This issue is not actionable, so I suggest closing, after leaving any comments for future searchers.

I'm slightly confused why we are continuing to create new proof purposes...

Which proof purpose are you talking about? The AssertionProofPurpose class uses the assertionMethod proof purpose which is what is in the VC data model spec.

I was referring to "CredentialIssuancePurpose:, It seemed I needed to explicitly specify AssertionProofPurpose and assertionMethod, why when verifying a VC would I need to specify AssertionProofPurpose, shouldn't that be the default?
I was able to resolve the issue, so feel free to close :)

I was referring to "CredentialIssuancePurpose:,

The CredentialIssuancePurpose class still uses the assertionMethod proof purpose, it just also runs the VC-specific check that ensures the specified issuer is the controller of the verification method.

It seemed I needed to explicitly specify AssertionProofPurpose and assertionMethod, why when verifying a VC would I need to specify AssertionProofPurpose, shouldn't that be the default?

This has been fixed (I believe) in #54. We just haven't pulled it in yet and done a new release.

just got bitten by this again using
"vc-js": "^0.6.1"
"jsonld-signatures": "^5.0.1",
Luckily I remembered it and came here...
This should really be handled internally, I'm not really aware of anyone creating VCs with purposes other than assertionMethod...
 const result = await vcjs.verifyCredential({
      credential: { ...verifiableCredential },
      suite: suite2,
      purpose: new AssertionProofPurpose(), // why is this not the default?
      compactProof: false,
      documentLoader,
    });

@OR13 I'm not sure what you're bumping into, but the default is assertionMethod. It is implemented through a class called "CredentialIssuancePurpose" but all that does is check for assertionMethod and that the controller of the verificationMethod used matches the issuer. So the default should be correct -- and our verifier code relies on it (it does not have to set anything explicitly).

Bitten again :)
"jsonld-signatures": "^5.1.0",
"vc-js": "^0.6.4"

It seems like this might be related to suites that are not shipped by default  with jsonld-signatures...
For example, this test fails when I use a custom suite and DO NOT provide the proof purpose...
import * as fixtures from '../__fixtures__';

const jsigs = require('jsonld-signatures');
const { Ed25519KeyPair } = require('crypto-ld');
const vc = require('vc-js');

const { Ed25519Signature2018 } = jsigs.suites;

const firstKey = fixtures.unlockedDid.publicKey[0];

const { documentLoader } = fixtures;

const key = new Ed25519KeyPair(firstKey);

const suite = new Ed25519Signature2018({
  key,
  date: '2019-12-11T03:50:55Z',
});

let verifiableCredential: any;
let verifiablePresentation: any;

const credential = {
  ...fixtures.credential,
  issuer: fixtures.unlockedDid.id,
  credentialSubject: {
    ...fixtures.credential.credentialSubject,
    id: fixtures.unlockedDid.id,
  },
};

jest.setTimeout(10 * 1000);

describe('vc-js-sanity', () => {
  it('issue verifiableCredential', async () => {
    verifiableCredential = await vc.issue({
      credential: { ...credential },
      suite,
      documentLoader,
    });
    expect(verifiableCredential.proof).toBeDefined();
  });

  it('verify verifiableCredential', async () => {
    const result = await vc.verifyCredential({
      credential: verifiableCredential,
      suite,
      documentLoader,
    });
    expect(result.verified).toBe(true);
  });

  it('createPresentation & signPresentation', async () => {
    const id = 'ebc6f1c2';
    const holder = 'did:ex:12345';
    const presentation = vc.createPresentation({
      verifiableCredential,
      id,
      holder,
    });
    expect(presentation.type).toEqual(['VerifiablePresentation']);
    verifiablePresentation = await vc.signPresentation({
      presentation,
      suite,
      challenge: '123',
      documentLoader,
    });
    expect(verifiablePresentation.proof).toBeDefined();
  });

  it('verify verifiablePresentation', async () => {
    const result = await vc.verify({
      presentation: verifiablePresentation,
      challenge: '123',
      suite,
      documentLoader,
    });
    expect(result.verified).toBe(true);
  });
});

If you want to reproduce, see https://github.com/w3c-ccg/lds-jws2020/blob/master/src/tests/vc-js-integration.spec.js#L132
I will likely figure this issue out when I implement vc-js from scratch some time in the future.

@dmitrizagidulin -- can you take a look here?

I should have more information in a bit.... it looks like i can change the name of the suite from JsonWebSignature2020 to Ed25519Signature2018 and suddenly all the tests pass. so I believe this has to do with how how types are registered and present in the 4+ contexts related to VCs with DIDs. (did context, credentials contexts (2), security-vocab)


API refactoring to allow management of non-Smalltalk code

This PR is pending since about 2 months. What do you want to do with it? There are new commits and the merge become harder day after day

meh... I will need to check at it :)

@npasserini I think I'm ready to integrate this... what do you say?


Define workflow for multiple conda environments

Motivation
Currently the conda environments are baked into the jupyterlab user image, dask image, and dask gateway image. As we build more complex conda environments the docker image size increases. Larger images mean longer startup times for each pod. At this point the 2-3 GB images are taking too long to start up. Additionally baking the conda environment into the image means that it is really not possible to have multiple environments easily.
Idea
Move these conda envirionments to a read only shared filesystem that is mounted into each users pod, worker, etc. This mount would take place over a ReadOnlyMany pvc over nfs. The downside of this approach is that we now have n pods all mounting the same nfs server ... meaning that we now need the throughput and iops of the 5-10 GB filesystem to be high. This can be expensive on cloud providers.
Requirements

develop some tool that can take a given list of envirionment yaml definitions

lock the packages in the given environment
archive the envrionement in a tarbal on a cheap storage (e.g. s3)
populate a fileysystem with n environements from an archive
updating the filesystem should be atomic (can only succeed or fail no partial update). symlinks likely our friend here

This approach would allow us to have a filesystem exactly reflect the state of n environment yaml files. Maybe this is a services, maybe a kubernetes job that is triggerd -- approach doesn't matter much but would be nice to have this independent of kuberentes since this could be a valuable tool that quansight could develop

Solved for now with conda-store and configuration file now supports multiple environments.


Segfault when running tensorflow.python.ops.gen_string_ops.string_format

Click to expand!
Issue Type
Bug
Have you reproduced the bug with TF nightly?
No
Source
binary
Tensorflow Version
2.11.0
Custom Code
Yes
OS Platform and Distribution
Ubuntu 22.04
Mobile device
No response
Python version
3.9
Bazel version
No response
GCC/Compiler version
No response
CUDA/cuDNN version
No response
GPU model and memory
No response
Current Behaviour?
Segmentation fault when feeding the following input combinations.

Standalone code to reproduce the issue
import tensorflow as tf
import os
import numpy as np
from tensorflow.python.ops import gen_string_ops
try:
  arg_0_0_tensor = tf.cast(tf.random.uniform([100, 100], minval=0, maxval=2, dtype=tf.int32), dtype=tf.bool)
  arg_0_0 = tf.identity(arg_0_0_tensor)
  arg_0 = [arg_0_0,]
  template = "{}"
  placeholder = ""
  summarize = False
  out = gen_string_ops.string_format(arg_0,template=template,placeholder=placeholder,summarize=summarize,)
except Exception as e:
  print("Error:"+str(e))

### Relevant log output

```shell
2023-01-21 19:56:19.403127: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2023-01-21 19:56:20.109647: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer.so.7'; dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: :/lib/:/home/nimashiri/anaconda3/envs/cuda11.2/lib/:/home/nimashiri/anaconda3/envs/cuda11.2/lib/
2023-01-21 19:56:20.109849: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer_plugin.so.7'; dlerror: libnvinfer_plugin.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: :/lib/:/home/nimashiri/anaconda3/envs/cuda11.2/lib/:/home/nimashiri/anaconda3/envs/cuda11.2/lib/
2023-01-21 19:56:20.109858: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.
2023-01-21 19:56:20.744012: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2023-01-21 19:56:20.749087: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2023-01-21 19:56:20.749223: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2023-01-21 19:56:20.749553: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2023-01-21 19:56:20.750231: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2023-01-21 19:56:20.750354: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2023-01-21 19:56:20.750461: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2023-01-21 19:56:21.186117: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2023-01-21 19:56:21.186259: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2023-01-21 19:56:21.186365: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2023-01-21 19:56:21.186458: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1613] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 4238 MB memory:  -> device: 0, name: NVIDIA GeForce GTX 1660 Ti, pci bus id: 0000:01:00.0, compute capability: 7.5
Segmentation fault

</details>

@nimashiri
You are sending invalid inputs to the API. Please note that the summarize attribute takes an integer value and prints the first and last summarized entries of each tensor when formatting the tensor summaries. Please find the gist  of working code here. Thank you!

@nimashiri You are sending invalid inputs to the API. Please note that the summarize attribute takes an integer value and prints the first and last summarized entries of each tensor when formatting the tensor summaries. Please find the gist of working code here. Thank you!

I am fuzz testing tensorflow backend.

@nimashiri, In tf-nightly, the above code is not causing Segfault. The summarize argument is considering False as 0 and True as 1 and producing valid outputs. Please find the gist here. Thank you!

@nimashiri, In tf-nightly, the above code is not causing Segfault. The summarize argument is considering False as 0 and True as 1 and producing valid outputs. Please find the gist here. Thank you!

Would you please run on TF-2.11 and confirm if you get segfaults?

If nightly does not show it it means it got resolved based on a report coming from elsewhere. On the next release, if the report is from external researcher, they will get credit.

@nimashiri,
When run on TF v2.11 the kernel is crashing with warnings, please refer to the image below

The above code is generating the expected output when using tf-nightly version 2.12.0.dev20230127. Please find the output below;
Output:
tf.Tensor(b'{}', shape=(), dtype=string)

Thank you!

It means the issue has been fixed. Most likely by https://github.com/tensorflow/tensorflow/commit/6d423b8bcc9aa9f5554dc988c1c16d038b508df1
Closing this as resolved.

It means the issue has been fixed. Most likely by 6d423b8
Closing this as resolved.

I am still getting segfault and Abort with the following test case:
import tensorflow as tf
import os
import numpy as np
from tensorflow.python.ops import gen_string_ops
try:
  arg_0_0_tensor = tf.cast(tf.random.uniform([10], minval=0, maxval=2, dtype=tf.int32), dtype=tf.bool)
  arg_0_0 = tf.identity(arg_0_0_tensor)
  arg_0 = [arg_0_0,]
  template = "{}"
  placeholder = "{}"
  summarize = 0.0
  out = gen_string_ops.string_format(arg_0,template=template,placeholder=placeholder,summarize=summarize,)
except Exception as e:
  print("Error:"+str(e))

Output:
2023-08-13 01:17:37.272600: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.
2023-08-13 01:17:38.161562: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2023-08-13 01:17:38.183248: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2023-08-13 01:17:38.183441: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2023-08-13 01:17:38.183987: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2023-08-13 01:17:38.184613: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2023-08-13 01:17:38.184759: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2023-08-13 01:17:38.184884: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2023-08-13 01:17:38.237798: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2023-08-13 01:17:38.237990: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2023-08-13 01:17:38.238127: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2023-08-13 01:17:38.238236: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1613] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 2530 MB memory:  -> device: 0, name: NVIDIA GeForce GTX 1660 Ti, pci bus id: 0000:01:00.0, compute capability: 7.5
Segmentation fault


Add bats test for certs plugin

Add tests for cert plugin, would catch this: #1658
Related PR discussion: https://github.com/progrium/dokku/pull/1667

:+1:


please look this worst issue

Analyzer Feedback from IntelliJ
Version information

IDEA AI-191.8<IP_ADDRESS>77832
2.7.0-edge.fa4379946109467c8a48f20f19d83d7c72968a3e
AI-191.8<IP_ADDRESS>77832, JRE 1.8.0_202-release-1483-b03x64 JetBrains s.r.o, OS Windows 10(amd64) v10.0 , screens 1920x1080

Exception
Dart analysis server, SDK version 2.7.0-edge.fa4379946109467c8a48f20f19d83d7c72968a3e, server version 1.27.4, FATAL error: Failed to handle request: {id: 4, method: analysis.setAnalysisRoots, params: {included: [E:\flutter\projects\vetapp], excluded: [E:\flutter\projects\vetapp\.idea, E:\flutter\projects\vetapp\.dart_tool, E:\flutter\projects\vetapp\.idea, E:\flutter\projects\vetapp\.pub, E:\flutter\projects\vetapp\build]}, clientRequestTime:<PHONE_NUMBER>172}
FileSystemException: Directory watcher closed unexpectedly, path = 'E:\flutter\projects\vetapp'
#0      _FileSystemWatcher._listenOnSocket.<anonymous closure> (dart:io-patch/file_patch.dart:309:11)
#1      _ExpandStream._handleData (dart:async/stream_pipe.dart:248:23)
#2      _ForwardingStreamSubscription._handleData (dart:async/stream_pipe.dart:164:13)
#3      _rootRunUnary (dart:async/zone.dart:1136:13)
#4      _CustomZone.runUnary (dart:async/zone.dart:1029:19)
#5      _CustomZone.runUnaryGuarded (dart:async/zone.dart:931:7)
#6      _BufferingStreamSubscription._sendData (dart:async/stream_impl.dart:336:11)
#7      _BufferingStreamSubscription._add (dart:async/stream_impl.dart:263:7)
#8      _SyncStreamControllerDispatch._sendData (dart:async/stream_controller.dart:764:19)
#9      _StreamController._add (dart:async/stream_controller.dart:640:7)
#10     _StreamController.add (dart:async/stream_controller.dart:586:5)
#11     new _RawSocket.<anonymous closure> (dart:io-patch/socket_patch.dart:1384:35)
#12     _NativeSocket.issueReadEvent.issue (dart:io-patch/socket_patch.dart:890:18)
#13     _microtaskLoop (dart:async/schedule_microtask.dart:41:21)
#14     _startMicrotaskLoop (dart:async/schedule_microtask.dart:50:5)
#15     _runPendingImmediateCallback (dart:isolate-patch/isolate_patch.dart:116:13)
#16     _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:173:5)

For additional log information, please append the contents of
file://C:\Users\amjad\AppData\Local\Temp\report.txt.

Thanks for reporting this issue! This was previously reported at https://github.com/dart-lang/sdk/issues/38853; please follow along there.

Duplicate of https://github.com/dart-lang/sdk/issues/38853


Mi mem clear intro

Thanks David for your PR. However, I don't feel a function like mem_clear belongs in an allocator library. We might need it some time in the future for a security heaps but now there is no clear case I think. Sorry.


Update attack-surface-reduction-rules-reference.md

Onenote shows  few hits on our ASR Audit
Added the same to the list.
Fixes#https://github.com/MicrosoftDocs/microsoft-365-docs/issues/10478

Learn Build status updates of commit ffff905:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

microsoft-365/security/defender-endpoint/attack-surface-reduction-rules-reference.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the learn.microsoft.com contributor guidesPost your question in the Learn support channel

Learn Build status updates of commit bb78b27:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

microsoft-365/security/defender-endpoint/attack-surface-reduction-rules-reference.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the learn.microsoft.com contributor guidesPost your question in the Learn support channel

Learn Build status updates of commit ac3c800:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

microsoft-365/security/defender-endpoint/attack-surface-reduction-rules-reference.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the learn.microsoft.com contributor guidesPost your question in the Learn support channel

Learn Build status updates of commit e13cd47:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

microsoft-365/security/defender-endpoint/attack-surface-reduction-rules-reference.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the learn.microsoft.com contributor guidesPost your question in the Learn support channel

Learn Build status updates of commit ca8bcd7:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

microsoft-365/security/defender-endpoint/attack-surface-reduction-rules-reference.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the learn.microsoft.com contributor guidesPost your question in the Learn support channel


Nested Accordion

I'm trying to have a nested accordion, is this possible ?
Thanks

Has anyone find a solution to this

Hi any luck with this?

It should be possible now by nesting the accordion in this


BUG: No amd64 build available for Ubuntu GUI download (only arm64)

Program version
3.0.7
Operating System
Linux
Type
Gui
Service
Irrelevant
Command used
N/A
Show ID
N/A
Episode
N/A
Console Output
N/A
Additional Information
There is no amd64 build available for the Ubuntu gui option download.  It's an arm64 build that is included and can't be installed on an amd64 version of Linux.  This seems to have changed starting 3.0.1 when you switched to electron builder.  I'm not sure if it was intentional or not.

I actually only have the most basic knowledge of Ubuntu.
Is it enough for you if I add the x64 architecture?

Yes.

The new default for ubuntu is now x64 (602d39e97b3695e00d33b5b6a12bd6b09418e7a1).
Arm can be built by cloning the repo and running the commands manually.


RTVE.es videos [rtve.alacarta] aren't fluid

Checklist

[x] I'm reporting a broken site support
[x] I've verified that I'm running youtube-dl version 2020.05.08
[x] I've checked that all provided URLs are alive and playable in a browser
[x] I've checked that all URLs and arguments with special characters are properly quoted or escaped
[x] I've searched the bugtracker for similar issues including closed ones

Verbose log
❯ youtube-dl -v --external-downloader aria2c --external-downloader-args '-c -j 3 -x 3 -s 3 -k 1M' --all-subs --embed-subs --postprocessor-args '-fflags +igndts' https://www.rtve.es/alacarta/videos/estoy-vivo/estoy-vivo-capitulo-1/4203918/
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'-v', u'--external-downloader', u'aria2c', u'--external-downloader-args', u'-c -j 3 -x 3 -s 3 -k 1M', u'--all-subs', u'--embed-subs', u'--postprocessor-args', u'-fflags +igndts', u'https://www.rtve.es/alacarta/videos/estoy-vivo/estoy-vivo-capitulo-1/4203918/']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2020.05.08
[debug] Python version 2.7.16 (CPython) - Darwin-19.3.0-x86_64-i386-64bit
[debug] exe versions: avconv 12.3, avprobe 12.3, ffmpeg 4.2.2, ffprobe 4.2.2, rtmpdump 2.4
[debug] Proxy map: {}
[rtve.es:alacarta] Fetching manager info
[rtve.es:alacarta] 4203918: Downloading JSON metadata
[rtve.es:alacarta] 4203918: Downloading url information
[rtve.es:alacarta] 4203918: Downloading m3u8 information
[rtve.es:alacarta] 4203918: Downloading subtitles info
[debug] Default format spec: bestvideo+bestaudio/best
[info] Writing video subtitles to: Estoy vivo - Temporada 1 - Capítulo 1-4203918.es.vtt
[debug] Invoking downloader on u'https://rtve-hlsvod.secure.footprint.net/resources/TE_GL23/mp4/4/2/1504716904424.mp4/1504716904424-audio_eng=96000-video_eng=1493000.m3u8?hls_minimum_fragment_length=6&hls_client_manifest_version=3'
[download] Destination: Estoy vivo - Temporada 1 - Capítulo 1-4203918.mp4
[download] 100% of 797.72MiB
[debug] ffmpeg command line: ffprobe -show_streams 'file:Estoy vivo - Temporada 1 - Capítulo 1-4203918.mp4'
[ffmpeg] Fixing malformed AAC bitstream in "Estoy vivo - Temporada 1 - Capítulo 1-4203918.mp4"
[debug] ffmpeg command line: ffmpeg -y -loglevel 'repeat+info' -i 'file:Estoy vivo - Temporada 1 - Capítulo 1-4203918.mp4' -c copy -f mp4 '-bsf:a' aac_adtstoasc -fflags '+igndts' 'file:Estoy vivo - Temporada 1 - Capítulo 1-4203918.temp.mp4'
[ffmpeg] Embedding subtitles in 'Estoy vivo - Temporada 1 - Capítulo 1-4203918.mp4'
[debug] ffmpeg command line: ffmpeg -y -loglevel 'repeat+info' -i 'file:Estoy vivo - Temporada 1 - Capítulo 1-4203918.mp4' -i 'file:Estoy vivo - Temporada 1 - Capítulo 1-4203918.es.vtt' -map 0 -c copy -map '-0:s' -map '-0:d' '-c:s' mov_text -map '1:0' '-metadata:s:s:0' 'language=spa' -fflags '+igndts' 'file:Estoy vivo - Temporada 1 - Capítulo 1-4203918.temp.mp4'
Deleting original file Estoy vivo - Temporada 1 - Capítulo 1-4203918.es.vtt (pass -k to keep)

Description
After post-processing, videos downloaded from RTVE Alacarta don't look fluid. This is due to the TS stream containing invalid PTS and ffmpeg correcting them in a way that results in non-fluid playback. More details can be found at this SO answer.
As the answer suggests, this can be fixed by using ffmpeg's igndts format flag. I'm trying to pass --postprocessor-args '-fflags +igndts' to youtube-dl but it's not helping, as ffmpeg options need to be specified before the file they apply on:

As a general rule, options are applied to the next specified file. Therefore, order is important, and you can have the same option on the command line multiple times. Each occurrence is then applied to the next input or output file. Exceptions from this rule are the global options (e.g. verbosity level), which should be specified first.

This is unfortunate, as output options only work when applied before the output file (but after the input file) and viceversa. We have no way to tell what the user wants, unless we introduce separate options for input args and output args.
My questions:

Would it make sense to introduce a --postprocessor-input-args flag?
How can we fix this for everyone using the RTVE extractor? Do we have a way to customise post-processing for specific websites?

Happy to help with PRs once we find out a way forward. Thanks!

I've never had problems with rtve videos, maybe is your ffmeg config? Tell me example

@felisucoibi nope, you can see the ffmpeg command in the debug output. Also, I have a local copy of youtube-dl patched to put the opts before the input files and it fixes the problem.
For a video example of the problem + an explanation of why it happens and a fix, see the Stack Overflow answer I linked to.

If you can give me a example of video of rtve with this problem, i don't have any problems and i downloaded a lot fo them. (or maybe i don't see it, please give a example)

I pasted the output of a download (including the URL) in the issue.
Alternatively, just try any episode of Estoy Vivo or El Ministerio del
Tiempo.
On Sun, May 24, 2020, 21:31 felisucoibi<EMAIL_ADDRESS>wrote:

If you can give me a example of video of rtve with this problem, i don't
have any problems and i downloaded a lot fo them. (or maybe i don't see it,
please give a example)
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/ytdl-org/youtube-dl/issues/25382#issuecomment-633283306,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAB5MIOV4RT6GSDNKGEBEQ3RTFYZ3ANCNFSM4NIUBLBQ
.

And you can see the problem when watching the video? i never noticed something strange in the video.

Have you tried my example? The playback it not fluid and almost trembling, like this example.
If you try to run the ffmpeg command manually, you'll see a lot of errors like these:
[mpegts @ 0x7fabfe802a00] Invalid timestamps stream=1, pts=21153600, dts=21160800, size=1936
[mpegts @ 0x7fabfe802a00] Invalid timestamps stream=1, pts=21160800, dts=21164400, size=2827
[mp4 @ 0x7fabff005000] Invalid DTS: 10800 PTS: 3600 in output stream 0:0, replacing by guess
[mp4 @ 0x7fabff005000] Invalid DTS: 14400 PTS: 10800 in output stream 0:0, replacing by guess

The reason is the RTVE stream has invalid presentation timestamps. Here is an explanation from someone who helped me fix the issue:

Frames in video streams with B-frames are not encoded or stored in presentation order, so the presentation timestamps have to be correctly offset for fluid playback. In this TS sample, you have presentation timestamps earlier than decoding timestamps, which are invalid, and ffmpeg defaults to assigning the DTS as the new PTS

This can be fixed by passing -fflags +igndts to ffmpeg before the input file, e.g.:
ffmpeg -fflags +igndts -i all.ts -c copy all.mp4

There is PR to fix this? some of the movies downloaded from rtve does not andthere are errors in the mkv probably due to this sync problem.


Metric verification job

This adds tests to verify metrics from all our metric sources are in prometheus.
rh-jira: https://issues.redhat.com/browse/OSPRH-9694

The job succeeded except for reporting the result, which I guess is gonna change a little? I'll get back to it in roughly 1.5 weeks.

I'd like to use the contents of https://github.com/infrawatch/feature-verification-tests/pull/157 to check for the ceilometer-compute-agent and node exporter containers on the computes. Afterwards this should be ready for another round of reviews.

I think this is now a genuine failure of getting the Node Exporter metrics and the job is rightly detecting the failure. The job confirms, that ScrapeConfig exists and that the NodeExporter container on the compute node is healthy, but there just aren't any metrics in the Prometheus. This is further confirmed by the list of metrics gathered by must-gather: https://logserver.rdoproject.org/44/144/7aec4340af741ecd5d8bccda2c8e74036a4d3dee/github-check/functional-metric-verification-tests-osp18/c0b985d/controller/ci-framework-data/logs/openstack-k8s-operators-openstack-must-gather/ctlplane/ceilometer/metric_list

recheck - the autoscaling fvt job failed on a tempest test??? tempest.scenario.test_network_basic_ops.TestNetworkBasicOps let's see if it passes on second try.

Recheck

recheck - issue should be fixed now


Add inspector warnings/helpers for PressableButton

When adding PressableButton to a game object, there are a number of things which must be true for it to perform correctly.  We should have inspector warnings and helpers to ensure that these things are true.

A NearInteractionTouchable or NearInteractionTouchableUnityUI must be present
The Touchable's EventsToReceive must be Touch
In the UnityUI case, there must be a MovingButtonVisuals present with a Graphic (aka, Image, RawImage, etc.) present

Closed via #6160


KMP Algorithm

💥 Proposal
KMP algorithm for pattern searching in c++
Please assign it to me;)

/assign


ES6 String interface lacks definition of 'includes'

The below code raise an compilation error, which says "Property 'includes' does not exist on type 'string'.", even with the "--target es6" option.
"abcde".includes("cd");

It is likely that "contains" function defined in String interface should be replaced with "includes".
https://github.com/Microsoft/TypeScript/blob/b2a871dcfb89739632fbc0fa5b5abe0177d68b9b/src/lib/es6.d.ts#L380
https://github.com/Microsoft/TypeScript/blob/965f994acefd238dc0c473e739d7916c47bf41b4/bin/lib.core.es6.d.ts#L1579

I found that this has been reported already in #2881.
Sorry for duplicate.


Using wildcards or params in the wrapper identifier

Snakemake version
8.6.0
Describe the bug
From the documentation: "It is possible to refer to wildcards and params in the wrapper identifier, e.g. by specifying "0.0.8/bio/{params.wrapper}" or "0.0.8/bio/{wildcards.wrapper}"."
However, this does not seem to be the case. I tried with a direct wildcard (e.g. {value}), {wildcards.value} and {params.wrapper} (having a parameter named wrapper) without success. The pipeline+wrapper work just fine when using a hardcoded string for the wrapper identifier.
Logs

Minimal example
When using the wildcard directly the --dry-run executes fine, but the real execution fails with:
    wrapper:
       "file:wrappers/tool/{value}/task/"

RuleException:
WorkflowError in file test_wrapper.snk, line 24:
Unable to locate wrapper script for wrapper file:wrappers/tool/{value}/task/. This can be a network issue or a mistake in the wrapper URL.

When trying to access {wildcards.value} or  {params.wrapper} it fails already in the --dry-run:
    params:
        wrapper="1.0.0"
    wrapper:
       "file:wrappers/tool/{params.wrapper}/task/"

WorkflowError:
Failed to open source file wrappers/tool/{params.wrapper}/task/environment.yaml
FileNotFoundError: [Errno 2] No such file or directory: 'wrappers/tool/{params.wrapper}/task/environment.yaml'

Additional context

I'm having the same problem with 8.16.0. It's a documented feature but it doesn't work you can't actually specify e.g. `{params.wrapper} or '{wildcards.wrapper}' in the wrapper directive!


removed custom webhooks authn

I'm starting it as a draft since it needs some more work.
@jaxoncreed do you think we can break up this document into a spec and a primer?

spec - would intentionally leave AuthN out of scope, only mentioning Solid-OIDC in a non-normative way.
primer - would show all the details and use specifically Solid-OIDC to provide complete guidance.

TODO

[ ] adjust conformance classes based on #120

#146 defines WebhookChannel2023
Once we merge it we can write a primer to accompany it.


Custom ROS 2 message definition files need a newline at the end for proper conversion

Description

Custom ROS 2 message definition files need a newline at the end for proper conversion

Steps To Reproduce

Record ROS 2 file on Galactic, and then try converting to MCAP in Humble

Expected Behavior

MCAP file is successfully created and deserialized in Studio

Actual Behavior

Created MCAP file uses incorrect message definitions, and is not able to be deserialized in Studio

closed in https://github.com/foxglove/mcap/pull/450


billing: Offer release announcement subscriptions.

Also avoid prompting for full time more than once.

Renumbered the migration and marked to merge once CI passes, thanks @amanagr and @karlstolley!

lgtm, marked to merge once CI passes.


[UX] Iconos Bancos MLA

Descripción
Se agregaron imágenes para los bancos:

Banco Municipal
Banco Chubut
Banco Saenz

Pruebas

Las imágenes se encuentran comprimidas
Card Issuers testeado

Codecov Report

Merging #684 into develop will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff            @@
##           develop     #684   +/-   ##
========================================
  Coverage    28.93%   28.93%
========================================
  Files          243      243
  Lines        17514    17514
  Branches      1520     1520
========================================
  Hits          5067     5067
  Misses       11922    11922
  Partials       525      525

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c065ef7...81083d3. Read the comment docs.


Add ensure_exists to resource

Add an option to check if the resource exists when creating a resource instance.
If the resource doesn't exist raise ResourceNotFoundError
Pod(name="nonexistspod", namespace="pod-namespace",ensure_exists=True)
ResourceNotFoundError: Resource `Pod` `pod-namespace/nonexistspod` does not exist

Report bugs in Issues
The following are automatically added:

Add reviewers from OWNER file (in the root of the repository) under reviewers section.
Set PR size label.
New issue is created for the PR. (Closed when PR is merged/closed)
Run pre-commit if .pre-commit-config.yaml exists in the repo.

Available user actions:

To mark PR as WIP comment /wip to the PR, To remove it from the PR comment /wip cancel to the PR.
To block merging of PR comment /hold, To un-block merging of PR comment /hold cancel.
To mark PR as verified comment /verified to the PR, to un-verify comment /verified cancel to the PR.
verified label removed on each new commit push.
To cherry pick a merged PR comment /cherry-pick <target branch to cherry-pick to> in the PR.

Multiple target branches can be cherry-picked, separated by spaces. (/cherry-pick branch1 branch2)
Cherry-pick will be started when PR is merged

To build and push container image command /build-and-push-container in the PR (tag will be the PR number).
To add a label by comment use /<label name>, to remove, use /<label name> cancel
To assign reviewers based on OWNERS file use /assign-reviewers
To check if PR can be merged use /check-can-merge

Supported /retest check runs

/retest tox: Retest tox
/retest python-module-install: Retest python-module-install

Supported labels

hold
verified
wip
lgtm

/verified


Changes to style sheets don't reload properly unless a JS file is also changed

Reproduction:

Create a test app: meteor create foo
Add css modules: meteor add nathantreid:css-modules
Add a class to main.css:

.foo {
  color: red;

Import the stylesheet and set up a helper in main.js:

import style from './main.css';
// ....
Template.hello.helpers({
  counter() {
    return Template.instance().counter.get();
  },
  style() {
    return style;
  },
});

Add the class in main.html:

<template name="hello">
  <button>Click Me</button>

You've pressed the {{style}} button {{counter}} times.

</template>

View the site in your browser. The result is as expected:

Change the color to green. View the site in your browser. The style disappears because the stylesheet object is no longer a css modules class name export but instead an HTMLStyleElement DOM object:

Make an arbitrary change in main.html or main.js. The style shows up as expected:

The Compiling undefined with meteor-babel without a cache error should be fixed in<EMAIL_ADDRESS>It was compiled using Meteor 1.7, so it may also resolve the SCSS issues you are having. I was able to use SCSS files in a simple test app.

@nathantreid Thank you so much, I confirm this is fixed.


Updated workflow to run test on pull request

Workflow created to run tests on pull requests for main and release branches

It's running the test twice because the "on" specifies both push and pull requests. Only need the action to run once

Gonna leave it for now and accept it.


I'm dumb, what is this?

that's pretty much it
I feel like this is something web or mobile related. Or maybe a way to run the port on older platforms?  Although ogl 1.3 seems old enough so maybe not.
I've done the WEB_TARGET=1 build on the port and tried what comes out.  It has .js files too but also a html and when I host it it makes sense.
But how do you run this?  or what do you do with it?

Hi @enigma9o7 ,
I don't think any of these are dumb questions, I expected some confusion here.
I took my website down temporarily, because I was warned about hosting Nintendo textures.  I just put it back up with the game removed for now, it explains many of the questions you are asking. But I'll answer them here as well.

What is this?

This is me rewriting / porting the C code manually into native Javascript

I've done the WEB_TARGET=1 build on the port and tried what comes out. It has .js files too but also a html and when I host it it makes sense.

Yep, that compiles the C code into web assembly.  And it works really well, probably works on as many devices and platforms as my port would, and probably at similar (or better?) performance (I don't really know tbh).  My motivation to make a native javascript port started because I wanted to build some multiplayer custom games and wanted to interface with the code more easily for that. But the main motivations for me anyways I list on that website Iinked above. In summary, I saw value in the rewriting the code in javascript or any higher level language because I believed it would open the door to more developers that may want to make custom content but aren't experienced in low level programming like C.  Additionally, even for people that are experienced in C, like myself, I started this project so I can more easily interface and add other software packages and tools that I may want to use to make other custom content.

But how do you run this? or what do you do with it?

I haven't shared instructions for this yet, but I will.  But I am using nodejs babel and webpack to build/compile the runnable javascript.   You'd need to install node and npm,  and then I think you'd just need to do something like
npm install npm run build
then that will put the index.html and index.js in the /dist folder and you can launch and run that.
or host the site by
node serveProduction.js
There might be other steps necessary, I'll look into that and see if I can make the process more simple

Build / Run Instructions have been added btw

Oops, didn't mean to mention this 👀


Log non-graceful termination to /var/log/kube-apiserver/termination.log and stdout

This adds a /var/log/kube-apiserver/termination.log file to masters (sitting next to the audit logs we already have). It lists non-graceful terminations for easier CI and customer cluster analysis.
Container logs alone are not enough because logging during termination is broken, and we lose logs from old pods. So we have to persist the data somewhere on disk.
Etcd is also no option as etcd struggles with the same termination issues and is not reliable.

/test e2e-gcp-upgrade

/retest

/retest

/retest

Flakes.
/retest

I really don't like nested signal termination in this case either - why wouldn't kube-apiserver summarize failure better on exit to logs? That's why brian and I added fallbacktologsonerror - so that our infra components could log better failures on exit?

Because we don't have logs. Logging of termination is broken in kubelet since 4.1. Our BZ is open since 4.1 too. Am happy to remove this again as soon as
a) kubelet and cri-o start working and doing their job
b) we have termination logs e.g. through loki.
We are spending half of our time hunting issues because we are blind. Waiting is no option.

/lgtm

/retest

/retest

/retest

/retest

/retest

/retest

/retest

Dependency merged.
/retest

/retest

/retest

/retest

/retest

/lgtm

ci-operator flake
/retest

/retest

/hold
failing ci/prow/e2e-aws is real, it looks like the must-gather test doesn't know how to handle terminating.gz files
 STEP: /tmp/test.oc-adm-must-gather.044028853/registry-svc-ci-openshift-org-ci-op-6nby4y5y-stable-sha256-90e78122d7d240f2b1a0eaa05507303205292cd745fbfa5b36298942dab81fe4/audit_logs/kube-apiserver/ip-10-0-163-93.us-west-2.compute.internal-.terminating.gz

https://github.com/openshift/origin/pull/25282 merged.
/retest

/hold cancel


Wrong syntax color for function args

When a function arg has a "complex" type (ie: (list 10 uint) or (string-utf8 10)), it breaks the following args colors.

I tried to tweak the regexps but didn't find a way to reliably fix it so far.
Maybe I'll try to come back on it later but I might need some guidance

I'll take care of it :wink:
New version (almost finished) will produce something like this:

Cool thanks! Looking forward to your PR.

Seems to be fixed 👍  Thx


Function to parse URIs

Repro is with Zed commit 59dc184.
A community user recently asked:

Is there any way to manipulate the data within a field?  For instance, I was looking to break apart the different parameters that are in a URI. /test/?first=sometext&second=moretext turning that into first=sometext and second=moretext

Regexp group support (#2093) has the potential to help here when we implement it. For now, I pointed out the existence of the split() function and slice operator that can accomplish much of this. For example:
$ zq -version
Version: v0.29.0-487-g59dc1843

$ echo '{"uri": "/test/?first=sometext&second=moretext"}' | zq -Z 'put qstring:=split(uri,"?")[1] | put pairs:=split(qstring,"&")' -
{
    uri: "/test/?first=sometext&second=moretext",
    qstring: "first=sometext&second=moretext",
    pairs: [
        "first=sometext",
        "second=moretext"
    ]
}

No doubt this could be taken even further by using the sequence operator to break out the entries in the pairs array into separate fields.
However, having shown this, I was also aware that there's probably escaping or other complexity that could make this approach break with some URIs. That made me search & find Go libraries like https://pkg.go.dev/net/url#example-URL.Query that purport to do this heavy lifting for us. Since URI manipulation is something that may come up frequently for our users, it might be beneficial for Zed to provide a URI-specific parsing function that returns a nested record containing the fully-parsed data.

Verified in Zed commit fdca08a.
Let's take an example Grafana URL that has lots of stuff to parse.
$ cat data.ndjson
{"url": "https://johndoe:easy2guess@play.grafana.org:443/d/000000010/annotations?orgId=1&from=now-3h&to=now"}

Calling the new function to parse it:
$ zq -version
Version: v0.30.0-13-gfdca08a7

$ zq -Z 'goodies := url_parse(url)' data.ndjson
{
    url: "https://johndoe:easy2guess@play.grafana.org:443/d/000000010/annotations?orgId=1&from=now-3h&to=now",
    goodies: {
        scheme: "https",
        opaque: null (string),
        user: "johndoe",
        password: "easy2guess",
        host: "play.grafana.org",
        port: 443 (uint16),
        path: "/d/000000010/annotations",
        query: |{
            {"to",[
                    "now"
                ]},
            {"from",[
                    "now-3h"
                ]},
            {"orgId",[
                    "1"
                ]}
        }|,
        fragment: null (string)
    }
}

Note that this is one of the first times a Zed function is returning a map-type value. Just to give a crash course on how to access a map's values, here we can make a top-level field that contains the value of one of the query parameters:
$ zq -Z 'goodies := url_parse(url) | put q_from := goodies.query["from"] | drop goodies' data.ndjson
{
    url: "https://johndoe:easy2guess@play.grafana.org:443/d/000000010/annotations?orgId=1&from=now-3h&to=now",
    q_from: [
        "now-3h"
    ]
}

If you try to access a key that isn't in the map, you get a warning, but it's otherwise a no-op.
$ zq -Z 'goodies := url_parse(url) | put q_from := goodies.query["absentparam"] | drop goodies' data.ndjson
put: a referenced field is missing
{
    url: "https://johndoe:easy2guess@play.grafana.org:443/d/000000010/annotations?orgId=1&from=now-3h&to=now"
}

A couple other things worthy of note:

The name of the function is likely to change to parse_url() (#3084)
The Brim app is presenting map values pretty crudely at the moment (https://github.com/brimdata/brim/issues/1245)

However, as something in Zed, this is definitely ready to show off to the user in the community that requested the functionality.
Thanks @nwt!


Update invalid cdn on angular-formly example page

Hi, it seems some of the example on http://angular-formly.com/#/ are not working due to wrong cdn address, making preview of example non-usable. I found 'ui-bootstrap tabs' example working when changed cdn of ui-bootstrap. There are also other few example which have similar/other problem. Please update these update these examples.

The examples seem to be working fine for me... Could you check them again?

As I wrote in formly-js/angular-formly-website#93
angular-ui-bootstrap v1.0.0 has a breaking change.

Thanks @kamilkisiela! :D

@kentcdodds I fixed that in few others examples.
https://github.com/formly-js/angular-formly-website/compare/ebd26ff...c1c2577
So if any other example uses angular-ui-bootstrap it should be fixed immediately

@kamilkisiela super! :+1:

Thanks for the fix and sorry for posting issue in wrong place :disappointed:


Add flake8-debugger

@charliermarsh, hi. I tried to bring the logic as close to the original as possible, but in debugger_call it was not completely successful. I will be glad to receive feedback.

Awesome! Will review this shortly. Thanks for putting this together.

@karpa4o4 - I did some small refactors to use match_call_path and friends, which are some utilities we have to do this kind of "Is a function call an invocation of a specific member, in a specific module?" logic.

But let me know if I messed anything up please :)

@charliermarsh, thanks. Sorry, I forgot to write "Closed" in the pull request description. So issue #546 stays open.
Can I help with the implementation of some other rule?

Oh good catch! Just closed the issue.

Can I help with the implementation of some other rule?

Definitely! There's a bunch of unclaimed stuff in https://github.com/charliermarsh/ruff/issues/827, if you're interested in those? Alternatively, there's https://github.com/charliermarsh/ruff/issues/458 but it's a bit more involved as I'd like it to be integrated with some of the existing docstring checks.

Oh, or https://github.com/charliermarsh/ruff/issues/870 -- we only need the first rule (U100). It'd probably look like the implementation we have for unused_variables, but we have to check for BindingKind::Argument.

@charliermarsh, hi. Unfortunately I could not do the issues that you described above. But now I want to participate in the development of ruff again, can you tell me how I can help?

No worries! If you're interested, you could consider working on the [tryceratops(https://github.com/charliermarsh/ruff/issues/2056) plugin.


[Azure ML SDK v2] Registering Data Asset from Job not working

Following this example I tried to register the output of a job as a Data Asset, by using the azureml:<my_data>:<version> syntax for the path. The following minimal example shows what I'm trying to achieve:
from azure.ai.ml import command
from azure.ai.ml.entities import Data
from azure.ai.ml import Input, Output
from azure.ai.ml.constants import AssetTypes

my_job_outputs = {
    "prep_data": Output(type=AssetTypes.URI_FOLDER, path="azureml:test_output:1")
}

job = command(
    command="echo 'test' > ${{outputs.prep_data}}",
    outputs=my_job_outputs,
    environment="test-env@latest",
    compute="cpu-cluster",
)

# submit the command
returned_job = ml_client.create_or_update(job)

Unfortunately, this fails. The job submission works, but the Job is immediately going to the Failed status with the following error message:
Invalid output uri azureml:test_output:1/ found for output prep_data of run , the list of supported uri formats are ["wasb://", "wasbs://", "adl://", "abfs", "abfss://", "azureml://"]
Could you clarify what the correct syntax is for registering a Data Asset from a Job output?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: c56bf00c-ec47-4a12-9a87-137fb29893e0
Version Independent ID: 86302e6a-7281-fb13-3702-63ea37622606
Content: Read and write data in jobs - Azure Machine Learning
Content Source: articles/machine-learning/how-to-read-write-data-v2.md
Service: machine-learning
Sub-service: mldata
GitHub Login: @ynpandey
Microsoft Alias: yogipandey

@tomasvanpottelbergh
Thanks for your feedback! We will investigate and update as appropriate.

@tomasvanpottelbergh
We have checked this issue, you have used for datastore path instead of data asset path that why you got this error. Kindly go through document and try again.

Please Note, GitHub forum is dedicated for docs related issues. For any technical queries or clarifications, we encourage to utilise Microsoft Q & A platform. Kindly raise your query on Microsoft Q&A Platform

Thank you for your answer @RamanathanChinnappan-MSFT. Could you please clarify, because I'm using exactly the syntax for data assets, as suggested in the docs and in your screenshot. I will also raise this on the Q&A Platform, but I think this is an issue with the docs, since the provided syntax does not work.

@tomasvanpottelbergh
Thanks for your feedback! We have assigned the issue to author and will provide further updates.

I got a same error here, following the same format azureml:<my_data>:<version>

@SturgeonMi
Please review it.

This document is not accurate. The supported formats for input and output are not the same.
If you want to specify the name and version of the output data asset, you can use name and version.
But path parameter for output doesn't support "azureml:<my_data>:".
The PR to modify the document is here: https://github.com/MicrosoftDocs/azure-docs-pr/pull/276903

@SturgeonMi @ynpandey I also tried to set name and version without path. But I got error: "ModelAssetPathNotFoundInStorage: No blobs found in storage at model asset path: azureml/6211e6d8-b49f-463b-8016-5b1d74ef304d/AzureMLModels/". Also another question, by only setting name and version, how can I distinguish outputting outputs to Models or Data?

You can set up type of input/output as mentioned in:
https://learn.microsoft.com/en-us/azure/machine-learning/how-to-read-write-data-v2?view=azureml-api-2&tabs=python
https://learn.microsoft.com/en-us/azure/machine-learning/how-to-manage-inputs-outputs-pipeline?view=azureml-api-2&tabs=cli#types-of-inputs-and-outputs

@tomasvanpottelbergh
We have checked this issue, you have used for datastore path instead of data asset path that why you got this error. Kindly go through document and try again.
Please Note, GitHub forum is dedicated for docs related issues. For any technical queries or clarifications, we encourage to utilise Microsoft Q & A platform. Kindly raise your query on Microsoft Q&A Platform

Using the datastore path instead of the data asset path worked, but is there a plan to make this feature more consistent? When defining a YAML pipeline, data asset paths work just fine. This is confusing when the only error that defining the pipeline programatically gives you is "asset not found."


HR-127 - [UI] Connect Login Page to Login API

Description

Connect login api when user clicks on Login button - API (useLoginMutation)
Add token to localstorage
Unsuccessful login will display error on login form.

Acceptance Criteria

Page will redirect to Home Page on successful login.
Unsuccessful login displays error on login page
token will be stored in localstorate
Unit test coverage should be at least 80%

This component is resolved with the branch login layout

This component is resolved with the login-layout branch


Bug in coffeeFor for loop generation code

"civet coffeeCompat"
for i in [0...n]
  5

transpiles to
var i
for (let i = i = 0, asc = 0 <= n; asc ? i < n : i > n; i = asc ? ++i : --i) {
  5
}

which crashes with i used before assignment.  If i is used within the body, it works: the second instance of i gets renamed to i1.  So maybe an issue with the ref mechanism?

It may be that the ref populating isn't checking the sub-expressions inside ForStatementParameters.

Investigating, it seems that the main issue is that this code on looks for Identifiers:
https://github.com/DanielXMoore/Civet/blob/507a0cb96bf6689a745f2c9c02127a801492e0eb/source/parser.hera#L5977-L5979
But in the AST we instead have this node:
    {
      type: 'AssignmentExpression',
      name: 'i',
      names: ['i'],
      children: [ { '$loc': [Object], token: 'i' } ]
    },

Should AssignmentExpression be changed to include an Identifier node, or should we add AssignmentExpression to the list of things to look for?

I think your first suggestion of having the generated AST nodes be totally conforming to the parsed node interfaces is the way to go.


Suggestion

Is it possible to add a feature that can automatically notice the users when there is an update in github?

感谢你的来信，在我看到邮件后我会尽快地回复你，我对对你邮件回复的延迟感到十分抱歉，感谢你的理解！Thank
you for your e-mail. I will reply as soon as I can if I can access to e-mails. I apologize for any delay in response to
your e-mail. Thanks!


Better handling of proxying shutdown

Before: if one end closes, we abruptly stop. We give confusing error
messages
After: if one end closes, we drain the other end then shutdown. Errors
are nicer.
The "Disconnected" case is as such: the client has sent us data, but the
server disconnected before we could send it (or the opposite).
This can actually happen in legitimate cases but its somewhat rare.
For instance, and HTTP server may send a response back immediately
without reading the request body and then close the connection.
fixes https://github.com/istio/ztunnel/issues/1065

In response to a cherrypick label: new pull request created: #1068


There is a problem with the service pages pattern (formally Add technical problem pages pattern)

Adds the technical problems page pattern, contributed by Steven Proctor.
https://trello.com/c/54ARFYk8/1020-get-technical-problem-page-contribution-ready-to-publish

You can preview this change here:
Built with commit 141a55d16472d76536830d6c1768bb817e289d35
https://deploy-preview-217--govuk-design-system-preview.netlify.com

Hey @stevenaproctor !
Would you mind reviewing this before it gets published? :)

@nickcolley I will be able to take a look tomorrow.

@nickcolley Looks good.


Play multiple YouTubePlayerViews at once

Question
Hi! I love this library. Is there a way to enable multiple video players playing at the same time? So that when I press play on a second player, the first one isn't paused? Thanks!

Hi, I think it should be doable, considering that the player is just running in a WebView. Unfortunately at the moment I don't really know how to do it.
Feel free to fork the library and explore, any contribution is welcomed :)


Windows XP: Cloudflare - endless download cycle

For about 2 weeks now I've been having problems accessing sites with Cloudflare protection.
There is an infinite loading cycle, but only on Windows XP.
On Windows 7 everything works: the checkout is normal.
Does anyone else have similar problems?
Windows XPSP3, Supermium_x86 124.0.6367.245 / 122.0.6261.152
For tests:
https://2captcha.com/demo/cloudflare-turnstile
https://www.inbox.lv/

Possible UACH discrimination issue. That's why Supermium will soon default to identifying as Google Chrome 128 running on Windows 11.

OFFTOP
and that is how the usage stats of various win versions, based on the browser' info, is falsified: no support and substitution of some info, started from v101/107 (but in same time such substitution can "secure" from well dumb targeted attacks)

Possible UACH discrimination issue.

This is unlikely. These checks pass normally as always. With the original user agent.

hm... can confirm that is not an CH issue. On Win XP SP2 x64 at least two errors come up that is not present on Win 7 SP1 x64
side-by-side XP and Win7 with swapped UA and UACH

yep there local rofl

and the second(actually first) error that happens before manual confirming

... My OS is Windows Vista SP2 32-bit;
On a Sm-122-r6 fresh profile (all settings and flags at default values, no extra extensions), I can easily pass both CF checks below:
https://2captcha.com/demo/cloudflare-turnstile-challenge
https://www.inbox.lv/
On my Sm-122-r6 "dirty" profile (a plethora of customised GUI settings, cmdline switches, internal flags, plus a number of installed extensions), I can finally pass those two CF tests above with greater difficulty and only after disabling certain pricacy/security related extensions, e.g.
Canvas Fingerprint Defender, Decentraleyes, Font Fingerprint Defender, I don't care about cookies, Privacy Badger, Simple Privacy Settings, uBlock Origin (development build), possibly even some Violentmonkey userscripts...

But a certain NSFW website,
https://rule34hentai.net/
will refuse to come through, either in a Sm-122-r6 pristine new profile or in an old "dirty" one :rage: ...
The CH "profile" of my Sm-122-r6 copy is shown below:

Moving on to Sm-124-final and in a minimal profile with just uBO-dev installed, the first two CF challenges are successfully passed quite easily :+1: , but when trying to load the NSFW site linked previously, I finally managed to pass its CF check (but not right away and only after I ticked the "verify you're human" box) ... The CH "profile" of my Sm-124 copy is:

It's absolutely outrageous that CF has become the gatekeeper of the internet, dictating how (OS, browser brand, browser version) one can access sites behind it - this is certainly material for the EFF and many more institutions...
It has become apparent to me that the inability to pass through CF protection is not a "one-size-fits-all" issue, this is controlled by a multitude of factors (even unique to different browser profiles of the same browser brand/major version) ...
While ClientHints are suspected as being the major villain here (and the subject of ad nauseam speculations on a dedicated MSFN thread), can the real experts here decide why Sm-124 can pass the CF check  at *hentai* while Sm-122 simply won't? Is it plain User-Agent-String, ClientHints or Javascript/CSS-engines related?

Thanks for any insight (BTW, that NSFW site was cited as non-functioning in another community, I simply use it here as one of the "difficult" cases; probably more exist :disappointed: ) ...

@Vangelis66
Unfortunately, I cannot do the test with your Operating System.
I would be very interested if this extension (you should use only this extension and not combine it with the others you use) solves your problem:
https://jshelter.org/
Obviously MV3 has reduced performance.

after looking at Vangelis66's screenshots i notice that his(and my) JS part not fully spoofed - browser version and platform from JS CHs differents from UA
and now, if i'm correct, i spoofed all possible places and even whatismybrowser stop telling me that i'm looks like Gecko while impersonated as Chrome plus runned on Win 0.6.0 or Win NT(just NT). And cloudflare demo from first post is passed as well while i'm impersonate my browser as <IP_ADDRESS> on Win XP x64 aka Srv 2003 and even XP x32
Chrome 122 @ Win XP x32 (impersonated, the real system is Win 7 x64)

so the real cause probably disordered and inconsistent info in CHs of JS, Headers and UserAgent(window.navigator.userAgent and window.navigator.appVersion) that may trigger some protection mechanisms
but on XP(with no extensions and on clean profile) there were some errors in browser console, so i check it later with correct spoofing and if it not solve the issue - assume there some issues on browser side at specific OS
p.s. now i'm thinking about own extension to do that all in one place instead of two extensions plus TM for JS part...

Possible UACH discrimination issue. That's why Supermium will soon default to identifying as Google Chrome 128 running on Windows 11.

Please don't change the user agent to Windows 11, since Windows 10 still has twice as many users: https://gs.statcounter.com/os-version-market-share/windows/desktop/worldwide Plus. I don't see any reason that if Supermium showed Win11 instead of Win10, it would change anything.

And one more important note: all old Firefox-based browsers (Palemoon, Serpent, Mypal) pass Cloudflare check for WindowsXP.
I have the problem only on chrome-like browsers (Supermium, Thorium, 360Extreme Explorer).

... That's probably because these forks do NOT support at all the Client Hints API :stuck_out_tongue_winking_eye: ; only relatively recent Chromium versios do...

as for 124 R2 the only CH that is not send yet is FormFactor, rest is due to config/flags. Even Arch and WoW-ness are sent

yep can confirm that turnstile test passed but real protection still not
Win XP SP2 x64; Supermium 126.0.6478.249 R3 x64
another website to test is https://sevenforums.com/

same here for Supermium 126.0.6478.249 R3 (not fresh R4) on x64 Win XP SP2 x64 , but with one difference: if you are using NoScript - you need to allow webgl for cloudflare.com, widely or for specific site - does not matter, important do so for cloudflare.com and not for target site. cloudflare.com had this permission all the time @ 2captcha.com, but, last time i checked it, verification was failed, now it's success.
screenshots, note - cf explicitly hide usage of webgl(no red highlighting)
webgl not allowed

webgl allowed


Mention the "after_commit" callback in the README

So that new developers can know that it exists, and know when it is called.

Feel free to submit a pull request.

Hi @rafaelfranca !
Thanks for the quick response.  I'll submit a PR, but I have one question first:  Should "after_commit" be called for an update that is stopped by the "dirty" module since there were no changes?  I would expect not, but I think I see this in at least one of our apps.
Just to clarify for the README change.

If no commits are made to the database it should not run. If it is happen seems like a bug.


Billing plane issue on the archived Nat-corder

Issue Description:
We are encountering crashes in both Unity Editor and our iOS app when attempting to record video using Nat-corder. This issue started occurring recently without any changes to the video capture functionality in our codebase.
Error Output:
NatCorder: MP4Recorder prepared video encoder at resolution<EMAIL_ADDRESS>with average bitrate 10000000 and keyframe interval 2s NatML billing plan only allows for limited functionality.
Trigger the record functionality that uses Nat-corder.
Observe Unity Editor crash or iOS app freeze and crash.

Additional Context:

The issue occurs regardless of recent changes to the codebase related to video recording.
Subscription status for NatML (Nat-corder’s dependency) has been verified and is active.
Attempting to catch the exception programmatically has not been successful, possibly due to the issue originating from the native side.

Proposed Solution:
Currently seeking advice or solutions on how to prevent the app from crashing when this exception occurs, or insights into potential causes from the Nat-corder or NatML community.

@BasilDavid we've migrated VideoKit over to its own service at videokit.ai. You can login with the same email as NatML then check your billing plan. Also, make sure to upgrade to VideoKit. We have discontinued NatCorder.


improved edgelist IO

Load edgelists using pandas; symmetrize adjacency matrix w/ scipy operations.
Reduction in load time can be significant -- on 10M edge subset of com-orkut.ungraph.txt, reduces load time from 422s to 25s (94% reduction in run time)
Passed regression tests on various sample datasets.

Can you comment on the change in memory requirements? One of the reasons for the old design was to minimize the extra memory required. (The switch to pandas reading is great!)
The benchmark is we'd like to do 350M nonzeros (i.e. twice the number of edges) in 16GB of memory.
I get a little worried with the manipulation of sparse matrices, which can easily balloon memory requirements.
It'd also be good to replace the symmetric check with the sum with a small number of random mat-vec products with A and A'. (I want to add a from_adjacency_matrix() option too, so this would get reused there...) It's very possible (due to common graph constructions) that you'd have equal row/column sums, but not actually be symmetric, this would protect us against that silly case. [Yes, I realize it's not full protection :) ]

Do you have an example of graph where you were going over 16GB using the previous method?  (The full ORKUT matrix is < 3GB as a CSR.)  Presumably a graph of that size would take many hours to load?
I'm not sure I understand what you mean about the randomized symmetry check.  I'm doing an exact symmetry check right now -- I personally wouldn't want to use a randomized algorithm unless I really had to (and I certainly wouldn't want to default to one).
FWIW, if you're only going to handle undirected graphs, I would just assume that the input graphs are properly formatted and maybe provide a utility for converting an arbitrary graph to the format you care about.  Then you could make these loading operations much faster and the cost of the initial transformation is amortized over all the subsequent local graph clustering runs.

Here is a small benchmark that I did
(Ben's) 9.4GB and 158 seconds
(Ben's without symmetry check) 5.83GB and 65 seconds
(Ours) 5.34GB and 2600 seconds
I think will use Ben's without the symmetric check.

Okay, great! we should add another issue to add a randomized symmetry check
to optimize that step.
In terms of Ben's reply (sorry, typing from email, so no fancy Github @),
my experience has been that NO one gets input formats correct. (Myself
included...) being robust to these inputs and converting them to our format
is essential, but we want to be as memory efficient as is reasonable.
On Wed, Mar 7, 2018 at 2:11 PM, kfoynt<EMAIL_ADDRESS>wrote:

Here is a small benchmark that I did
(Ben's) 9.4GB and 158 seconds
(Ben's without symmetry check) 5.83GB and 65 seconds
(Ours) 5.34GB and 2600 seconds
I think will use Ben's without the symmetric check.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/kfoynt/LocalGraphClustering/pull/36#issuecomment-371244993,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AACEywwn2KekNAuXTMU-tBXn5UWMTPrjks5tcC0zgaJpZM4SgvRr
.

Hi David,
I brought back symmetry actually for the reason that you mentioned. I do not trust the users.
Also, even if they do add the correct input the total memory requirement would have been about the same.
I think our previous solution using list_to_CSR was probably not ready for wide use because it was very slow, although very cool!
Maybe if we implement this in C this will speed up list_to_CSR so it is comparable to Ben’s solution?
I would prefer that if possible because it has half the memory requirements, i.e., 5GB vs 10GB.
Kimon

On Mar 7, 2018, at 11:56 AM, David Gleich<EMAIL_ADDRESS>wrote:
Okay, great! we should add another issue to add a randomized symmetry check
to optimize that step.
In terms of Ben's reply (sorry, typing from email, so no fancy Github @),
my experience has been that NO one gets input formats correct. (Myself
included...) being robust to these inputs and converting them to our format
is essential, but we want to be as memory efficient as is reasonable.
On Wed, Mar 7, 2018 at 2:11 PM, kfoynt<EMAIL_ADDRESS>wrote:

Here is a small benchmark that I did
(Ben's) 9.4GB and 158 seconds
(Ben's without symmetry check) 5.83GB and 65 seconds
(Ours) 5.34GB and 2600 seconds
I think will use Ben's without the symmetric check.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/kfoynt/LocalGraphClustering/pull/36#issuecomment-371244993,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AACEywwn2KekNAuXTMU-tBXn5UWMTPrjks5tcC0zgaJpZM4SgvRr
.

—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub https://github.com/kfoynt/LocalGraphClustering/pull/36#issuecomment-371264333, or mute the thread https://github.com/notifications/unsubscribe-auth/ALLf2cjACVf7cW-iOzEuTqcb3dTYSLj_ks5tcDuKgaJpZM4SgvRr.

Yes -- definitely agree that no one gets the input formats correct.  And definitely shouldn't trust the users 😄
I generally prefer checking the format and throwing an error if there's something wrong a) so that expensive checks don't have to happen every time the graph is loaded and b) because for very large graphs I often do the preprocessing in Spark or something else with a lot of horsepower.  Also, converting a directed graph to an undirected one sometimes involves some kind of domain-specific transform (should you take max weight? min weight? add the weights? etc.)
Up to you guys how you'd like to handle it though -- LMK if there are any changes to the code you need from me.
~ Ben

Hi Ben,

I generally prefer checking the format and throwing an error if there's something wrong

Even if we do this the memory problem will to transferred to a different place, i.e., the data file.
Since it is generally accepted that in an edgelist for undirected data we do not store all edges I assume that users
would expect their input to have this property. So, I will stick for now with your symmetry check + symmetrization.

Also, converting a directed graph to an undirected one sometimes involves some kind of domain-specific transform (should you take max weight? min weight? add the weights? etc.)

This happens currently only for gml or graphml. They that it is handled is based on networkx:
https://networkx.github.io/documentation/networkx-1.10/reference/generated/networkx.Graph.to_undirected.html https://networkx.github.io/documentation/networkx-1.10/reference/generated/networkx.Graph.to_undirected.html
Kimon

On Mar 7, 2018, at 12:29 PM, Ben Johnson<EMAIL_ADDRESS>wrote:
Yes -- definitely agree that no one gets the input formats correct. And definitely shouldn't trust the users 😄
I generally prefer checking the format and throwing an error if there's something wrong a) so that expensive checks don't have to happen every time the graph is loaded and b) because for very large graphs I often do the preprocessing in Spark or something else with a lot of horsepower. Also, converting a directed graph to an undirected one sometimes involves some kind of domain-specific transform (should you take max weight? min weight? add the weights? etc.)
Up to you guys how you'd like to handle it though -- LMK if there are any changes to the code you need from me.
~ Ben
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub https://github.com/kfoynt/LocalGraphClustering/pull/36#issuecomment-371273922, or mute the thread https://github.com/notifications/unsubscribe-auth/ALLf2XiRoXoqilMTIPjGBjrO4bUvHX8Nks5tcEMjgaJpZM4SgvRr.

Sorry for the typos!! :-D

On Mar 7, 2018, at 12:35 PM, Kimon Fountoulakis<EMAIL_ADDRESS>wrote:
Hi Ben,

I generally prefer checking the format and throwing an error if there's something wrong

Even if we do this the memory problem will to transferred to a different place, i.e., the data file.
Since it is generally accepted that in an edgelist for undirected data we do not store all edges I assume that users
would expect their input to have this property. So, I will stick for now with your symmetry check + symmetrization.

Also, converting a directed graph to an undirected one sometimes involves some kind of domain-specific transform (should you take max weight? min weight? add the weights? etc.)

This happens currently only for gml or graphml. They that it is handled is based on networkx:
https://networkx.github.io/documentation/networkx-1.10/reference/generated/networkx.Graph.to_undirected.html https://networkx.github.io/documentation/networkx-1.10/reference/generated/networkx.Graph.to_undirected.html
Kimon

On Mar 7, 2018, at 12:29 PM, Ben Johnson<EMAIL_ADDRESS><EMAIL_ADDRESS>wrote:
Yes -- definitely agree that no one gets the input formats correct. And definitely shouldn't trust the users 😄
I generally prefer checking the format and throwing an error if there's something wrong a) so that expensive checks don't have to happen every time the graph is loaded and b) because for very large graphs I often do the preprocessing in Spark or something else with a lot of horsepower. Also, converting a directed graph to an undirected one sometimes involves some kind of domain-specific transform (should you take max weight? min weight? add the weights? etc.)
Up to you guys how you'd like to handle it though -- LMK if there are any changes to the code you need from me.
~ Ben
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub https://github.com/kfoynt/LocalGraphClustering/pull/36#issuecomment-371273922, or mute the thread https://github.com/notifications/unsubscribe-auth/ALLf2XiRoXoqilMTIPjGBjrO4bUvHX8Nks5tcEMjgaJpZM4SgvRr.


lib/pq v1.10.3 possibly fixes connection timeout issue?

Hello there,
I saw that the lib/pq project just released version v1.10.3. I seems that this release introduced the context.WithTimeout functionality which should close a connection after a certain duration.
If so pq-timeouts could help people to migrate their code back to raw lib/pq.
Let me know what are your thoughts on this.

@jacobfederer I don't think recent changes to lib/pq addresses the safeguards put in place by pq-timeouts.
From my understanding, pq-timeouts ensures Read/Write calls to the underlying tcp connection are preceded by SetReadDeadline/SetWriteDeadline to prevent them from hanging.
lib/pq doesn't seem to use any of those functions still. Only SetDeadline is used for dial.
For example it just wraps the conn in a *bufio.Reader and invokes io.ReadFull when reading.
About the cancellation, I think the v1.10.3 release notes you linked refer to this PR.
lib/pq uses watchCancel functions (1 & 2) to start this cancellation sequence when the context gets cancelled. It uses a context with a hardcoded 10 seconds timeout.
I think there's an edge case when you have no connect_timeout set, which is that DialContext is used but no deadlines are set afterwards. See here.
As the DialContext doc says:
Once successfully connected, any expiration of the context will not affect the connection.

Which means that the subsequent Write done in sendStartupPacket call could very well block.
There's an even weirder edge case that doesn't necessarily apply here: If for some reason you use DialOpen with a Dialer that does not implement DialerContext, you may very well block on the dial itself since the context with the 10 seconds timeout is not used.


lakefs on KV - run benchmarks on a constant data set

Per @arielshaqed comments on #3377 - the KV benchmark shoulld not use b.N to control the size of the data set (should be unifrom for all values of b.N, per a given test)

One of my main concerns is that benchmarks that rely on a certain dataset size, will not be able to scale with b.N, e.g. what should be the size of the dataset to allow b.N deletes?!
Discussed this wilts @nopcoder and the suggestion is to make a sampling size K to run b.N times. Each iteration (out of b.N) will create a dataset of size K, reset timers and run K operation
for n := 0; n < b.N; n++ {
	createDataSetOfSize(K)
	b.ResetTimer()
	deleteAllEntriesInDataSet
}


Clean up Examples notebooks

Clean up the examples notebooks to ease the burden on the user as they first begin to use LUCI.
Things to be done include (but are not limited to):

make sure file and folder paths are properly appended to the project repo path
consider using os.path as the default to get the user's current directory and remove the need to add it
Increased explanation on functionality and results - I will be a good test user for this piece to know if more info should be there :)

Will do! The examples were intended for people who already have a fair amount of familiarity with SITELLE and the results, so I will work on rewording them.

I've changed the example data to be even easier! Is that what you had in mind? @jgcovi

taking a look

@jgcovi How are they looking?

I think the main example notebook looks good now. Easy to get the data and I can follow it pretty well. If anything, the graph might need to be cleaned up since it's cutting off the title and I can't seem to fix it directly from the figsize or anything, but it's not detrimental. We can close this


EnableMeltdownCompatFlag no longer needed

According to https://support.microsoft.com/en-us/help/4072699/windows-security-updates-and-antivirus-software:

Our recent work with our anti-virus (AV) partners on compatibility with Windows updates has now reached a sustained level of broad ecosystem compatibility.  Based on our analysis of available data, we are now lifting the AV compatibility check for the March 2018 Windows security updates for supported Windows 10 devices via Windows Update.  We continue to require that AV software is compatible and in cases where there are known issues of AV driver compatibility, we will block those devices from updates to avoid any issues. We recommend customers check with their AV provider on compatibility of their installed AV software product.

Maybe you want to keep the flag around anyway. I don't know. I just wanted to let you know about this update :).

Yeah, security updates for supported Windows 10 devices sounds rather ambiguous. The tweak is commented by default anyway, so it doesn't hurt to keep it around for a little bit longer.
Thanks for the info.


No errors for inserts

Hello. How to get error from ClickHouse during INSERT?
Error: Request failed with status code 400

What have you tried so far?

I have the same problem

@maxpain @KeHuangT
Can you please share the code on how you are trying to catch errors?

I also faced this issue. I was trying to insert an ISO format date in a DateTime column. There was a generic axios error - 'Error: Request failed with status code 400' . It took me a while to figure out the real error.

@ApurvGpt @maxpain @KeHuangT
should be fixed


Use ellipse minor axis as diameter in morphometrics

Following discussions on issues #347  and #349 , we want to implement a new feature for computing morphometrics.
This feature should:

Use the minor axis of a fitted ellipse as axon diameter and axon+myelin diameter.
Give the choice to the user to compute morphometrics with the current method (equivalent diameter) or the new method (minor axis).
Integrate the change in FSLeyes (should we open a separate issue?)

As a reference, the minor_axis_length of the ellipse is already computed in skimage.measure.regionprops we use for axon and axonmyelin objects properties, example here.


mail1.io

Temp mail provider by def24, has a selector for their hostnames on the main page.
Current DEA domains include:
basketball.group
britishpreschool.net
busykitchen.coms
hacker9.org
l9pdev.com
linux0.net
ucm8.com
looksecure.net
mail1.local
megaradical.com
multiscanner.org
pimmel.top
privacylock.net

thanks, added as 'mail1.io'


Partial zoom

Hello,
We are using leaflet in an application, and my boss demands that I find a way to implement partial zoom - say in .2 increments instead of 1.  I googled around trying to find a way, but the best I found was a comment in 2012, where Vladimir, a maintainer said that it was too much work. That is not really an option for me, as my boss says that a big client demands this.
https://leaflet.uservoice.com/forums/150880-ideas-and-suggestions-for-leaflet/suggestions/2839764-continuous-fluid-zoom
I have looked through the source.  Does anybody have any suggestions on where to start?  If I get it figured out I may be able to submit a pr.
THANKS!

Fractional zoom is already supported in beta1, but there are further improvements coming soon — see https://github.com/Leaflet/Leaflet/pull/3523

Hallelujah!  Thanks a million!


Latest releases (1.23.x, 1.24.x) missing from NuGet. Invalid 2.x versions show up

None of these can be found in NuGet

Also there are 2.x versions that seem to overlap wit grpc-dotnet versions.
This is highly confusing. Dependency updates are suggested for 2.x releases and after upgrade we're seeing
 ---> System.TypeInitializationException: The type initializer for 'Google.Api.AnnotationsReflection' threw an exception.
 ---> System.IndexOutOfRangeException: Index was outside the bounds of the array.
   at Google.Protobuf.Reflection.ExtensionCollection.<>c__DisplayClass2_0.<.ctor>b__0(FieldDescriptorProto extension, Int32 i)
   at Google.Protobuf.Reflection.DescriptorUtil.ConvertAndMakeReadOnly[TInput,TOutput](IList`1 input, IndexedConverter`2 converter)
   at Google.Protobuf.Reflection.ExtensionCollection..ctor(FileDescriptor file, Extension[] extensions)
   at Google.Protobuf.Reflection.FileDescriptor..ctor(ByteString descriptorData, FileDescriptorProto proto, IEnumerable`1 dependencies, DescriptorPool pool, Boolean allowUnknownDependencies, GeneratedClrTypeInfo generatedCodeInfo)
   at Google.Protobuf.Reflection.FileDescriptor.BuildFrom(ByteString descriptorData, FileDescriptorProto proto, FileDescriptor[] dependencies, Boolean allowUnknownDependencies, GeneratedClrTypeInfo generatedCodeInfo)
   at Google.Protobuf.Reflection.FileDescriptor.FromGeneratedCode(Byte[] descriptorData, FileDescriptor[] dependencies, GeneratedClrTypeInfo generatedCodeInfo)
   at Google.Api.AnnotationsReflection..cctor()
   --- End of inner exception stack trace ---

Also there are 2.x versions that seem to overlap wit grpc-dotnet versions.
This is highly confusing. Dependency updates are suggested for 2.x releases and after upgrade we're seeing
 ---> System.TypeInitializationException: The type initializer for 'Google.Api.AnnotationsReflection' threw an exception.
 ---> System.IndexOutOfRangeException: Index was outside the bounds of the array.
   at Google.Protobuf.Reflection.ExtensionCollection.<>c__DisplayClass2_0.<.ctor>b__0(FieldDescriptorProto extension, Int32 i)
   at Google.Protobuf.Reflection.DescriptorUtil.ConvertAndMakeReadOnly[TInput,TOutput](IList`1 input, IndexedConverter`2 converter)
   at Google.Protobuf.Reflection.ExtensionCollection..ctor(FileDescriptor file, Extension[] extensions)
   at Google.Protobuf.Reflection.FileDescriptor..ctor(ByteString descriptorData, FileDescriptorProto proto, IEnumerable`1 dependencies, DescriptorPool pool, Boolean allowUnknownDependencies, GeneratedClrTypeInfo generatedCodeInfo)
   at Google.Protobuf.Reflection.FileDescriptor.BuildFrom(ByteString descriptorData, FileDescriptorProto proto, FileDescriptor[] dependencies, Boolean allowUnknownDependencies, GeneratedClrTypeInfo generatedCodeInfo)
   at Google.Protobuf.Reflection.FileDescriptor.FromGeneratedCode(Byte[] descriptorData, FileDescriptor[] dependencies, GeneratedClrTypeInfo generatedCodeInfo)
   at Google.Api.AnnotationsReflection..cctor()
   --- End of inner exception stack trace ---

Is caused by https://github.com/protocolbuffers/protobuf/issues/6749

Starting from "1.23", gRPC C# made a major version bump and became v2.23.0
The background and the rationale for that decision is described here: https://github.com/grpc/proposal/blob/master/L57-csharp-new-major-version.md

@jtattermusch ok, but why close when the last two releases are still missing from NuGet?

They are not missing. v2.23 and v2.24 are the C# nugets that correspond to the 1.23 and 1.24 branches in this repository. There are several languages that live in this repository and we only have one release process (we release packages for all languages in this repository at the same time) so C# 2.23 is actually part of the 1.23.x release train.
Additional context is here:
https://groups.google.com/forum/#!msg/grpc-io/tCTlDjG_Vsg/NlRZVMMqAQAJ

@jtattermusch perhaps I'm missing something, then.
The additional context mentions:

The release process and the release cadence for gRPC C# stays unchanged (which means that 2.23.x will be released at the same time as the 1.23.x version of other languages, gRPC C# 2.24.x will be released along with 1.24.x etc.)

In grpc i see:

1.24.0
1.24.1
1.24.2

Can you point to the corresponding 2.24.x releases in NuGet?

Ah, I didn't realize you were asking about the 2.24.1 and 2.24.2 patch releases specifically - these hasn't been published for C# yet. I'll check if those patches have anything that's relevant to C# (it might not) and push the nugets if needed. Until then you can use 2.24.0 version from nuget (which does exist https://www.nuget.org/packages/Grpc.Core/2.24.0)


changes for convolution symmetry

I started to fix https://github.com/scipy/scipy/issues/5897 before I realized it had already been done in https://github.com/scipy/scipy/pull/5904, so these are the other changes I made in the process.
(I was going to change slice_obj = [slice(None, None, -1)] * len(kernel.shape) into kernel[::-1] before I remembered it's N-dimensional, so I added a comment and renamed the variable to clarify that for others.)
real_array.conj() does nothing and just returns real_array, faster than
iscomplexobj(real_array) tests whether it's real, so I changed it to just call .conj() in both cases.
a = rand(4)

a.conj() is a
Out[105]: True

also changed a bunch of len(var.shape) into var.ndim.

@@            master   #5909   diff @@
======================================
  Files          238     238
  Stmts        43592   43590     -2
  Branches      8179    8178     -1
  Methods          0       0
======================================
- Hit          34020   34018     -2
  Partial       2602    2602
  Missed        6970    6970

Review entire Coverage Diff as of 4e6d72d

Powered by Codecov. Updated on successful CI builds.

+1 for the changes.  This is probably minor, but merging mine before yours seems slightly preferable since rebasing your changes onto my PR would be a little easier than the other way IMHO.

LGTM, +1 for merge other than my comment. @endolith you okay with taking one for the team and waiting for @gfyoung's PR to land so you can rebase?

added the comment.
I don't care when it gets merged :)
also I could rebase it on top of @gfyoung 's PR and make it part of that PR

Rebasing immediately is probably fine, though I'm hoping my PR can get merged in soon and then it will be easier (at least in my mind) for you to rebase onto master instead. :smile:

added some more things to make sure they aren't forgotten about

Merged into https://github.com/scipy/scipy/pull/5904

Looks like these didn't make it into https://github.com/scipy/scipy/pull/5904 so I'll reopen this PR

(also making some other changes...)

I don't know when this was introduced, but there's a discrepancy in handling of complex conjugates in scipy 0.17.0:
    scipy.signal.correlate(1j, 2j)              (-2+0j)
    scipy.signal.correlate([1j], [2j])          array([ 2.+0.j])
    scipy.signal.correlate([[1j]], [[2j]])      array([[ 2.+0.j]])
    scipy.signal.correlate2d([[1j]], [[2j]])    array([[-2.+0.j]])
    numpy.correlate([1j], [2j])                 array([ 2.+0.j])

Wikipedia's definition of cross-correlation is

which would mean conjugating the 1j into -1j and then multiplying it to get +2 as the correct output.

@endolith : Not really sure why your changes didn't get merged in when my PR was merged.  Sorry about that!  Your efforts did not go unappreciated though!

LGTM, thanks @endolith !


Python Snippet's Play and Play/Inspect buttons get disabled (greyed out) when an error occurs

Python Snippet's Play and Play/Inspect buttons get disabled (greyed out) when an error occurs depending on the one pressed.
Only reloading the Lepiter page fixes this.
Command-G continues to work though.
This can be prevented by switching the debugger view from Python to Pharo before closing it.

I've run into this as well

Related to this is the indication in the top right (Python @ address:port).
It always seems to lag behind, or is plain wrong.
At least something like 'Reset PythonSnippet' would be useful to get back in a good state.


lint the osx and windows apps

This also fixes a trivial bug in the osx release.

merging as trivial


use blackbox written together consistently

I noticed that in some places blackbox was used and in others black box. But it does raise the question if some rules should have aliases, since one can see also black-box and black box being used.

As a copyeditor I am slightly ashamed to have missed those inconsistencies. Thank you much @lsmith77 for identifying that.
Having an alias or alternate spellings section seems like a good idea, especially around these combination words that may not even have a standard spelling at this point (black-box & blackbox vs room-mate & roommate, latter is settled.) I'll look at the commit and see where to go from here.

@quaid thx for the merge .. I noticed another consistency issue with capitalization, ie. Tribe vs. blackbox


Add ability to set tab size for output/preview or use the setting from the editor

Hi @ThomasEnssner, can you elaborate on this? The number of spaces inside of the editor will be carried over to the preview automatically. Shiki does not have a "tab size" setting.


fix typo in pod-security-policy.md

This change is

@crewjam can you fix merge conflict please? PR looks good. Thanks!
/assign

Seems like the paragraph this PR touches has been removed, so I guess this PR is no longer needed. @crewjam please check it :)

Section that had the typo was removing, closing this PR since its no longer relevant


Refactor Model::ApplyClientConnectionChanges

Description
Refactor Model::ApplyClientConnectionChanges from:
ApplyClientConnectionChanges(connectionID string, changeFunc func(*ClientConnection)) *ClientConnection

to:
ApplyClientConnectionChanges(clientConnection *ClientConnection, changeFunc func(*ClientConnection))

So it overrides *clientConnection with updated value and possibly inserts new ClientConnection in the model if there is no one with the same ConnectionID.
Motivation and Context
Existing ApplyClientConnectionChanges doesn't update local variable. It is confusing.
How Has This Been Tested?

[x] Covered by existing integration testing
[ ] Added integration testing to cover
[x] Tested locally
[ ] Have not tested

Types of changes

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)

Checklist:

[ ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.

Looks like It is not a part of your pull request but can you consider refactoring baseDomain to use SyncMap instead using map and sync.RWMutex?

@denis-tingajkin
There is an issue if we use SyncMap instead using map and sync.RWMutex : https://github.com/networkservicemesh/networkservicemesh/pull/1158#discussion_r287937791

@Bolodya1997 I thought baseDomain has only simple operations on the map. Yep, you are right we can't use sync.Map in this place

Depends on #979.

@haiodo shall we consider this one obsolete already?

Closing, since we will do a complex NSM refactoring for SDK.


Drag-n-drop: support OS DND events coming in and going out of window

probably also best way to support between-window DND, rather than trying to cobble a custom solution.  with all the existing clipboard etc support it shouldn't be too hard, at least on mac. :)

need to test external drop in V2, and should be relatively easy to support between-window DND among our apps, which would go a long way (perhaps that is as far as we go)

I did the demo test in imgui, just export the window variables, how it should work in gogi
https://github.com/ddkwork/imgui/blob/master/example/example.go#L31-L35

3f8965105 implements the external drop event, supported in filetree, so you can drop a file into the filetree.
There is an issue with updating the filetree for such events so that needs to be fixed as well -- I'll file an additional ticket for that.