cannot connect to the marketplace

Issue Type: Bug
when searching anything from marketplace there is an error message "We cannot connect to the marketplace now, please try again later".
What is definite

marketplace was working some days ago
working environment is unchanged (OS, libs, network, etc) except for normal extension updating
my computer is not behind any proxy
network works and dns is ok (same version and setup VSC on another PC using the same network works normally)
GFW is not the culprit (either Google, Youtobe, or Github, Twitter are accessible, and marketplace homesite can be opened with browser)

Here is the log screenshot got under developer console

Here is the setup info

In addition, I have reviewed some old posts such as
https://github.com/microsoft/vscode/issues/81115
almost the same issue, there was no resolution in detail though it was said "solved" and closed, therefore I have to post it again, sorry for this.

here are some supplements:

extension update works normally
extension recommendation works normally

but still cannot connect to extension gallery

update:
as mentioned above, VSC with the same version and setup installed on another PC behind the same network works normally, I have compared the script file in which the error occurred as logged to the normal one. Surprisingly, they are absolutely the same, not a bit different.

update 2:
I have reviewed some similar issues submitted by others and found this image link below,
https://ms-python.gallerycdn.vsassets.io/extensions/ms-python/python/2019.8.29288/1565127378610/Microsoft.VisualStudio.Services.Icons.Default
and yes, it is accessible in my browser.

@pizzacrystal May I know what is the response of the marketplace request, you can find it in the network tab

@sandy081
Thanks for your question.
the first figure below shows network responses right after starting up of VSC and clicking extension button. the second figure below illustrates responses when i search something on searching box of marketplace (surrounded by red rect).

Seems network requests are good.
Which version are you on? Please try on latest and insiders?

I just updated VSC to latest version, here it is

but this issue still exists
I will try insiders as soon as i go back home, thanks!

Experiencing this same issue.
Version: 1.40.0 (user setup)
Commit: 86405ea23e3937316009fc27c9361deee66ffbf5
Date: 2019-11-06T17:02:13.381Z
Electron: 6.1.2
Chrome: 76.0.3809.146
Node.js: 12.4.0
V8: 7.6.303.31-electron.0
OS: Windows_NT x64 10.0.17763

I am not on any insiders build. Same symptoms:

Can update existing extensions
Can install recommended extensions
CANNOT search any other extensions

@bestekov Are you also seeing the same error as in description?
@pizzacrystal Please confirm after updating.

@sandy081 , where would I go to find those error logs?

@sandy081
sorry for updating so late
yes, i tried insiders but no lucky
and i have updated to the latest release version, the issue is still there.
版本: 1.40.1 (system setup)
提交: 8795a9889db74563ddd43eb0a897a2384129a619
日期: 2019-11-13T16:49:35.976Z
Electron: 6.1.2
Chrome: 76.0.3809.146
Node.js: 12.4.0
V8: 7.6.303.31-electron.0
OS: Windows_NT x64 10.0.18362

@pizzacrystal
Can you please show the response of the request extensionsQuery below
https://user-images.githubusercontent.com/11605580/68596524-58599600-04d6-11ea-94dc-df19b38be84a.png

@sandy081
do you mean this?

@sandy081
i don't know why but the 'extensionquery' item never shows up. please check below,

Try typing in the search box and you shall see the request.

@sandy081
here the response is, for your convinience, I put them into a file.

response.txt

Response looks good.
I am running out of ideas what would have caused it.

Please open shared process dev tools and see if there are any errors. Use command Developer: Toggle Shared Process
Try to reproduce by launching VS Code with different user data directory from command line. code --user-data-dir <path-to-user-data-dir>
Try to reproduce by launching VS Code with different extensions directory from command line. code --extensions-dir <path-to-extensions-dir>

@sandy081

no error shows up in "shared process" window

GOOD NEWS, extension gallery can be accessed again with tmp user data directory.
Therefore, I made some attempts further --
go to my user data folder, DELETE "cache" sub folder under it, and restart vscode again, extension gallery came back!

I am not clear why it works, frankly i don't care, but here is my question :
Is there any side effect as I deleted "cache" folder? Though as far as now, I did not find any one. Thanks!

This cache folder is from chromium in electron on which VS Code is built on. I guess there won't be any major side affects.
Thanks for confirming.

For me, the issue was related to the following files (Windows).
C:\Users\USERNAME\AppData\Roaming\Code\User\globalStorage\state.vscdb
C:\Users\USERNAME\AppData\Roaming\Code\User\globalStorage\state.vscdb.backup

I quit Code, and renamed them.
C:\Users\USERNAME\AppData\Roaming\Code\User\globalStorage\state.vscdb-disabled
C:\Users\USERNAME\AppData\Roaming\Code\User\globalStorage\state.vscdb.backup-disabled

Code recreated the files, and I was able to search the extensions again.


Clicking get info with a CSV/GACS locks the fields

This'll be fixed in the next version but comment here if it's a big priority for you and I'll push it out a little faster


Fix console output readability

This fixes issue #73

@Jorg3Lucas can you please check and merge this fix as this is really annoying. Reading console output is really hard without this fix! Thanks!


[BUG] Error "exits with return code -7" when finetuning FLANT5-xxl on 8x A100

1. Bug Description
I am finetuning Flan-T5-xxl with my corpus using DeepSpeed, based on the tutorial. But when I execute 'deepspeed --num_gpus=8 scripts/run_seq2seq_deepspeed.py', when the GPUs have loaded all shards, the process has immediately been terminated, exits with return code -7.
Here are the full console outputs:
$ deepspeed --num_gpus=8 run_seq2seq_deepspeed.py
[2023-02-24 18:10:18,983] [WARNING] [runner.py:186:fetch_hostfile] Unable to find hostfile, will proceed with training with local resources only.
[2023-02-24 18:10:19,049] [INFO] [runner.py:548:main] cmd = /home/aiops/xxxx/.miniconda3/envs/torch-py39/bin/python -u -m deepspeed.launcher.launch --world_info=eyJsb2NhbGhvc3QiOiBbMCwgMSwgMiwgMywgNCwgNSwgNiwgN119 --master_addr=<IP_ADDRESS> --master_port=29500 --enable_each_rank_log=None run_seq2seq_deepspeed.py
[2023-02-24 18:10:22,043] [INFO] [launch.py:142:main] WORLD INFO DICT: {'localhost': [0, 1, 2, 3, 4, 5, 6, 7]}
[2023-02-24 18:10:22,043] [INFO] [launch.py:148:main] nnodes=1, num_local_procs=8, node_rank=0
[2023-02-24 18:10:22,043] [INFO] [launch.py:161:main] global_rank_mapping=defaultdict(<class 'list'>, {'localhost': [0, 1, 2, 3, 4, 5, 6, 7]})
[2023-02-24 18:10:22,043] [INFO] [launch.py:162:main] dist_world_size=8
[2023-02-24 18:10:22,043] [INFO] [launch.py:164:main] Setting CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7
Loading checkpoint shards: 100%|██████████████████████████████████████████████████████████████████████████| 5/5 [00:34<00:00,  6.91s/it]
Loading checkpoint shards: 100%|██████████████████████████████████████████████████████████████████████████| 5/5 [00:37<00:00,  7.49s/it]
Loading checkpoint shards: 100%|██████████████████████████████████████████████████████████████████████████| 5/5 [00:40<00:00,  8.11s/it]
Loading checkpoint shards: 100%|██████████████████████████████████████████████████████████████████████████| 5/5 [00:41<00:00,  8.37s/it]
Loading checkpoint shards: 100%|██████████████████████████████████████████████████████████████████████████| 5/5 [00:43<00:00,  8.60s/it]
Loading checkpoint shards: 100%|██████████████████████████████████████████████████████████████████████████| 5/5 [00:43<00:00,  8.74s/it]
Loading checkpoint shards: 100%|██████████████████████████████████████████████████████████████████████████| 5/5 [00:44<00:00,  8.95s/it]
Loading checkpoint shards: 100%|██████████████████████████████████████████████████████████████████████████| 5/5 [00:45<00:00,  9.10s/it]
[2023-02-24 18:12:41,354] [INFO] [comm.py:657:init_distributed] Initializing TorchBackend in DeepSpeed with backend nccl
Using cuda_amp half precision backend
[2023-02-24 18:12:42,235] [INFO] [logging.py:68:log_dist] [Rank 0] DeepSpeed info: version=0.8.0, git-hash=unknown, git-branch=unknown
[2023-02-24 18:13:10,284] [INFO] [launch.py:318:sigkill_handler] Killing subprocess 17786
[2023-02-24 18:13:10,284] [INFO] [launch.py:318:sigkill_handler] Killing subprocess 17787
[2023-02-24 18:13:10,286] [INFO] [launch.py:318:sigkill_handler] Killing subprocess 17788
[2023-02-24 18:13:10,287] [INFO] [launch.py:318:sigkill_handler] Killing subprocess 17789
[2023-02-24 18:13:10,620] [INFO] [launch.py:318:sigkill_handler] Killing subprocess 17790
[2023-02-24 18:13:10,953] [INFO] [launch.py:318:sigkill_handler] Killing subprocess 17791
[2023-02-24 18:13:10,954] [INFO] [launch.py:318:sigkill_handler] Killing subprocess 17792
[2023-02-24 18:13:11,287] [INFO] [launch.py:318:sigkill_handler] Killing subprocess 17793
[2023-02-24 18:13:11,901] [ERROR] [launch.py:324:sigkill_handler] ['/home/aiops/xxxx/.miniconda3/envs/torch-py39/bin/python', '-u', 'run_seq2seq_deepspeed.py', '--local_rank=7'] exits with return code = -7

2. Screenshots

here is the console cursor before being killed:

[2023-02-24 18:12:42,235] [INFO] [logging.py:68:log_dist] [Rank 0] DeepSpeed info: version=0.8.0, git-hash=unknown, git-branch=unknown

here are the GPU and CPU states screenshots before being killed:

3. ds_report output
--------------------------------------------------
DeepSpeed C++/CUDA extension op report
--------------------------------------------------
NOTE: Ops not installed will be just-in-time (JIT) compiled at
      runtime if needed. Op compatibility means that your system
      meet the required dependencies to JIT install the op.
--------------------------------------------------
JIT compiled ops requires ninja
ninja .................. [OKAY]
--------------------------------------------------
op name ................ installed .. compatible
--------------------------------------------------
async_io ............... [NO] ....... [OKAY]
cpu_adagrad ............ [NO] ....... [OKAY]
cpu_adam ............... [NO] ....... [OKAY]
fused_adam ............. [NO] ....... [OKAY]
fused_lamb ............. [NO] ....... [OKAY]
quantizer .............. [NO] ....... [OKAY]
random_ltd ............. [NO] ....... [OKAY]
sparse_attn ............ [NO] ....... [OKAY]
spatial_inference ...... [NO] ....... [OKAY]
transformer ............ [NO] ....... [OKAY]
stochastic_transformer . [NO] ....... [OKAY]
transformer_inference .. [NO] ....... [OKAY]
utils .................. [NO] ....... [OKAY]
--------------------------------------------------
DeepSpeed general environment info:
torch install path ............... ['/home/aiops/xxxx/.miniconda3/envs/trm-pt-py39/lib/python3.9/site-packages/torch']
torch version .................... 1.13.1+cu116
deepspeed install path ........... ['/home/aiops/xxxx/.miniconda3/envs/trm-pt-py39/lib/python3.9/site-packages/deepspeed']
deepspeed info ................... 0.8.0, unknown, unknown
torch cuda version ............... 11.6
torch hip version ................ None
nvcc version ..................... 11.2
deepspeed wheel compiled w. ...... torch 1.13, cuda 11.6

4. System Info

OS: Ubuntu 18.04
GPU: A100 (40G each) x8
CPU x60, Mem 500G

5. Solution? Help!
I understand that error code -7 does not mean OOM (OOM is -9). I've searched over the whole Internet while failing to get the clue of error code -7 deepspeed.
So please help, what is the meaning of error code -7? And how to solve this and make the code training smoothly?

@scofield7419, did you pass a ds_config on the command line? Can you share the contents of your ds_config? Thanks!

Hi @tjruwase, yes, in my trials, I used ds_flan_t5_z3_config_bf16.json and ds_flan_t5_z3_offload_bf16.json respectively as it is without any modification. But both results in the same issue, error code -7.

Any update on this issue @scofield7419? :)

@scofield7419 Could you check your NCCL version and make sure it is greater than v2.10? BF16 support requires a NCCL version above 2.10.

Hi @jomayeri, thanks for the advice, but my cluster has an NCCL version of 2.10.3, is this working for the version requirement?

@scofield7419 Could you check your NCCL version and make sure it is greater than v2.10? BF16 support requires a NCCL version above 2.10.

I have the same problem.

Hi @tjruwase, I'm using 500G memory, with only 1-2 batch size. I tend not to believe this is the OOM problem. Also I tested with FLANT5-xl smaller one, and ended up with the same error.

I have the same issue too when working on pythia, it works with fp16, but when switched to bf16, it errors out with -7 without showing any other meaningful error. The model is only 1.5B and I have 8 80G A100s. It's definitely not OOM. NCCL version is also correct. So I believe something is wrong in deepspeed.

Here is the ds config that doesn't work:
    "bf16": {
        "enabled": "auto"
    },
    "optimizer": {
        "type": "AdamW",
        "params": {
            "lr": "auto",
            "betas": "auto",
            "eps": "auto",
            "weight_decay": "auto"
        }
    },

    "scheduler": {
        "type": "WarmupLR",
        "params": {
            "warmup_min_lr": "auto",
            "warmup_max_lr": "auto",
            "warmup_num_steps": "auto"
        }
    },

    "zero_optimization": {
        "stage": 3,
        "offload_optimizer": {
            "device": "cpu",
            "pin_memory": true
        },
        "offload_param": {
            "device": "cpu",
            "pin_memory": true
        },
        "overlap_comm": true,
        "contiguous_gradients": true,
        "sub_group_size": 1e9,
        "reduce_bucket_size": "auto",
        "stage3_prefetch_bucket_size": "auto",
        "stage3_param_persistence_threshold": "auto",
        "stage3_max_live_parameters": 1e9,
        "stage3_max_reuse_distance": 1e9,
        "stage3_gather_16bit_weights_on_model_save": true
    },

    "gradient_accumulation_steps": "auto",
    "gradient_clipping": "auto",
    "steps_per_print": 2000,
    "train_batch_size": "auto",
    "train_micro_batch_size_per_gpu": "auto",
    "wall_clock_breakdown": false
}

Did you solved it? I meet the same problem.

ds_config for fp16 also doesn't work

same error! save me!

same error!

Setting the shm-size to a large number instead of default 64MB when creating docker container solves the problem in my case. It appears that multi-gpu training relies on the shared memory.

I am having the same problem, for me the training in running on Azure VM 4 T4 GPU, but when I am trying to containerize and run in container I am getting this exit code.

@Shrishml Try increasing the shm-size of the container as suggested in the comment above.

I ran with this aks cluster yaml
`apiVersion: apps/v1
kind: Deployment
metadata:
name: training-deployment
namespace: genai
labels:
app: training
spec:
replicas: 1
selector:
matchLabels:
app: training
template:
metadata:
labels:
app: training
spec:
nodeSelector:
compute: gpu
containers:
- name: training
image: acrgenerative.azurecr.io/optimus-train:latest
volumeMounts:
- mountPath: /dev/shm
name: dshm
imagePullPolicy: Always
volumes:
- name: dshm
emptyDir:
medium: Memory
imagePullSecrets:
- name: optimussecret
`
or docker command docker run --rm --runtime=nvidia --gpus all --shm-size 3gb imagename
it worked

Increasing the shared memory size of the docker container seems to resolve the issue.

Hi @scofield7419 , did you end up finding a solution for this? Running into the same issue.

@jomayeri is there a solution for this if I am not using a docker container?

@chaitanyamalaviya Please open a new bug with the requested information and assign it to me and I'll take a look.

Done, created an issue here.


Distributed actors: Remote call crash with a function having generic protocol with associated type

Description
Basically this is a copy of an issue in swift-distributed-actors repo:
https://github.com/apple/swift-distributed-actors/issues/1156
I've just realised, that this could be a Swift runtime issue.
This is very specific, but still a bit surprising.
When using a protocol with associated type, which will be used then as a generic (see reproducible for example)—cluster will crash with error:
Thread 1: Fatal error: Error raised at top level: DistributedCluster.GenericRemoteCallError(message: "Remote call error of [ExecuteDistributedTargetError] type occurred")

Reproduction
you need ClusterSystem package (swift-distributed-actors)
import Distributed
import DistributedCluster

typealias DefaultDistributedActorSystem = ClusterSystem

protocol SomeProtocol: Codable {
  associatedtype Message: Codable
}

struct Implementation: SomeProtocol {
  typealias Message = String
}

distributed actor WorkerActor {

  static var key: DistributedReception.Key<WorkerActor> { "worker_actor" }

  distributed func getSome<S: SomeProtocol>(_ protocol: S, message: S.Message) -> String {
    "help"
  }

  init(actorSystem: ClusterSystem) async {
    self.actorSystem = actorSystem
    await self.actorSystem.receptionist.checkIn(self, with: WorkerActor.key)
  }
}

distributed actor ReceptionistActor {

  var actors: [WorkerActor] = []

  func listen() {
    Task {
      for await actor in await actorSystem.receptionist.listing(of: WorkerActor.key) {
        self.actors.append(actor)
      }
    }
  }

  distributed func getLatest() -> WorkerActor? {
    self.actors.last
  }

  init(actorSystem: ActorSystem) async {
    self.actorSystem = actorSystem
    self.listen()
  }
}

let receptionist = await ClusterSystem("receptionist")
let someActorsNode = await ClusterSystem("spme_actors") { settings in
  settings.bindPort = 1111
}

receptionist.cluster.join(endpoint: someActorsNode.settings.endpoint)
try await receptionist.cluster.joined(node: someActorsNode.cluster.node, within: .seconds(10))

let receptionistActor = await ReceptionistActor(actorSystem: receptionist)
let worker = await WorkerActor(actorSystem: someActorsNode)
// local actor works
try await print(worker.getSome(impl, message: "hello"))
// wait a bit for receptionist
try await Task.sleep(for: .seconds(3))
let remoteActor = try await receptionistActor.getLatest()
let impl = Implementation()
// Remote actor will crash
try await print(remoteActor?.getSome(impl, message: "hello") ?? "")
print("done")

Stack dump
* thread #1, queue = 'com.apple.main-thread', stop reason = Fatal error: Error raised at top level: DistributedCluster.GenericRemoteCallError(message: "Remote call error of [ExecuteDistributedTargetError] type occurred")
  * frame #0: 0x00000001b02b1890 libswiftCore.dylib`_swift_runtime_on_report
    frame #1: 0x00000001b0370b10 libswiftCore.dylib`_swift_stdlib_reportFatalErrorInFile + 208
    frame #2: 0x00000001aff42d94 libswiftCore.dylib`closure #1 (Swift.UnsafeBufferPointer<Swift.UInt8>) -> () in closure #1 (Swift.UnsafeBufferPointer<Swift.UInt8>) -> () in Swift._assertionFailure(_: Swift.StaticString, _: Swift.String, file: Swift.StaticString, line: Swift.UInt, flags: Swift.UInt32) -> Swift.Never + 104
    frame #3: 0x00000001aff41ed8 libswiftCore.dylib`Swift._assertionFailure(_: Swift.StaticString, _: Swift.String, file: Swift.StaticString, line: Swift.UInt, flags: Swift.UInt32) -> Swift.Never + 260
    frame #4: 0x00000001affe3628 libswiftCore.dylib`swift_errorInMain + 636
    frame #5: 0x000000010000cd38 WorkingPoolTest`async_MainTY19_ at <compiler-generated>:0
    frame #6: 0x000000010000cdc0 WorkingPoolTest`thunk for @escaping @convention(thin) @async () -> () at <compiler-generated>:0
    frame #7: 0x000000010000cee8 WorkingPoolTest`partial apply for thunk for @escaping @convention(thin) @async () -> () at <compiler-generated>:0

Expected behavior
Remote actor call should behave the same as when local with associated types in protocol.
Environment
swift-driver version: <IP_ADDRESS> Apple Swift version 5.10 (swiftlang-<IP_ADDRESS> clang-15<IP_ADDRESS>)
Target: arm64-apple-macosx14.0
Additional information
Note—everything will start working again if you'll change associated type to separate protocol, like:
protocol SomeProtocol: Codable {}
protocol Message: Codable {}
extension String: Message {}

and update function accordingly:
distributed func getSome<S: SomeProtocol, M: Message>(_ protocol: S, message: M) -> String {
    "help"
}

Thank you for the report! It is definitely possible we don't handle this well


neon: Add shl_n and shlq_n intrinsics

Should I have called the files shl_n?
However the shl and shlq intrinsics can still be added later.

I am not sure why formatting couldn't find shl.h.
The other tests have obviously found it.

Could you hold off merging this now please?
I hope I have nearly finished the shr_n and shrq_n intrinsics and I shall add them here.

I have added the shr_n and shrq_n intrinsics but I named the files without the _n.
Others can be added later but I don't think that the strange vshl intrinsics (which also shift right) translate well into other SIMD instruction sets.

OK, I have finally understood what formatting is saying. I originally thought it was a spurious message saying that shl.h didn't exist. I only just realized that it was saying it wasn't included by neon.h.
Shouldn't the test files include neon.h instead of e.g. shl.h so that this error is detected early?

Just to say that the AppVeyor failure isn't my fault.
I can't see how to restart it without sending a dummy commit :-(

Shouldn't the test files include neon.h instead of e.g. shl.h so that this error is detected early?

I want to make it possible to include only the functions you need for NEON if you want.  If we want that feature, this is really the only option… if we include neon.h we can't really test that any dependencies have been included (e.g., if you're implementing vpmin with vmin, you need to include min.h).
The CI check is there to make sure you don't forget to add it to arm/neon.h, so we cover both cases.

The CI check is there to make sure you don't forget to add it to arm/neon.h, so we cover both cases.

Yes, but IMHO it arrives too late: it is not detected either by the local tests or by the Travis CI tests.
You only realize after issuing a PR leading to wasted time and wasted Travis (and other) CI resources.

I understand, but with what you propose we lose the ability to test only
including what you need, and of the two I think that's the more error-prone.
Maybe the formatting checks should be in a script you can run locally?
You'd still have to remember to run it, but you could add that as a
pre-commit hook...
On June 12, 2020 10:03:21 rosbif<EMAIL_ADDRESS>wrote:

The CI check is there to make sure you don't forget to add it to
arm/neon.h, so we cover both cases.
Yes, but IMHO it arrives too late: it is not detected either by the local
tests or by the Travis CI tests.
You only realize after issuing a PR leading to wasted time and wasted
Travis (and other) CI resources.—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or unsubscribe.

I was thinking about implementing the vshl and vshlq macros next (those without the _n_).
This will complete shl.h (shr.h is already complete).
(Remark: there are also other shift instructions but they have different mnemonics.)
I was waiting for you to merge these _n_ ones first but should I continue on top of this PR as I did for shr.h?

Sorry, I got a bit behind on merging PRs this week.
I'm going to move this to shl_n.h.  My original intent was to actually use shl.h for all the different versions, but after @seanptmaher submitted the dup_n stuff in its own file I found I quite liked the idea.  I think it should make it a little easier to manage, especially when copying new a file over to start working on a new function.  We can always merge them in the future if the number of files gets out of hand, but I don't think it should be a problem.
BTW, I was really just thinking about making C-Reduce easier to use, but I thought of another major advantage of using insn.h instead of neon.h; incremental compilation should be a lot faster since the build system doesn't need to recompile everything; just the object files which use the headers that changed, then re-link eveything.
Anyways, in the future if I get behind on the PRs you can always just keep going and I'll figure out how to merge it.  I figure if the merge gets messy because I was slow that's on me, you shouldn't have to worry about it.

One minor nitpick: the macros to prevent multiple inclusion should also have N, particularly for shl_n as they will conflict with the logical name of the future file containing the missing shl intrinsics.
AFAICS they don't.

One minor nitpick: the macros to prevent multiple inclusion should also have _N, particularly for shl_n as they will conflict with the logical name of the future file containing the missing shl intrinsics.
AFAICS they don't.

Ah, you're absolutely right, thanks ☺
I noticed a few other things while I was reviewing this; I already changed them, but for future reference:

The parameter names should match the documentation.  There were a ton of b variables that are called n in the NEON docs.  Obviously it's b works, but it's nice to be consistent with the docs.
The shift amounts don't need to be int32_t; int is appropriate here.  AFAICT there is no need to worry about the width of these parameters, since the value is never more than 64 (63?).
I move a bunch of implementations from macros into the functions.  I know you thought that the x86 ones required an immediate-mode parameter, but even if that were true the WASM ones don't, there is no need to make them macros for defensive purposes.  This actually revealed a couple of minor bugs regarding types because now the input has to be compatible with int, which is then cast to unsigned char/short/int/long long in the function.

I feel good about this CI run on on my other repo, I think I'll have this merged into master in an hour or so.

OK, I agree that the number of files should remain reasonable but, if it does get out of hand, I am not volunteering to merge a significant number of them ;-)

No problem, I'll take care of it.  It should actually be really easy; splitting them up would be a bit of a pain, but merging seems pretty straightforward.  The only problem I see is that it would break backwards compatibility for the whole only include what you need idea.  Maybe I should add a guard against including the _n versions directly and just tell people to include the  "main" header for each instruction…?

I noticed a few other things while I was reviewing this; I already changed them, but for future reference:
* The parameter names should match the documentation.  There were a ton of `b` variables that are called `n` in the NEON docs.  Obviously it's `b` works, but it's nice to be consistent with the docs.

Sorry, I think the b's were a leftover from the example I copied, probably add.h.

* The shift amounts don't need to be `int32_t`; `int` is appropriate here.  AFAICT there is no need to worry about the width of these parameters, since the value is never more than 64 (63?).

Agreed.

* I move a bunch of implementations from macros into the functions.  I know you thought that the x86 ones required an immediate-mode parameter, but even if that were true the WASM ones don't, there is no need to make them macros for defensive purposes.  This actually revealed a couple of minor bugs regarding types because now the input has to be compatible with int, which is then cast to unsigned char/short/int/long long in the function.

I still maintain that these require an immediate-mode parameter: the IIG even gives the assembler instruction which should be generated. While current compilers seem to work around this limitation, there is absolutely nothing preventing a future compiler from conforming strictly with the IIG. So I don't really agree, but it is your call.
Sorry for the bugs :-(

I feel good about this CI run on on my other repo, I think I'll have this merged into master in an hour or so.

OK, I agree that the number of files should remain reasonable but, if it does get out of hand, I am not volunteering to merge a significant number of them ;-)

No problem, I'll take care of it. It should actually be really easy; splitting them up would be a bit of a pain, but merging seems pretty straightforward. The only problem I see is that it would break backwards compatibility for the whole only include what you need idea. Maybe I should add a guard against including the _n versions directly and just tell people to include the "main" header for each instruction…?

I don't think that there will be a "main" header for shr. AFAICS shr is already complete as all shr instructions are _n.

Sorry, I think the b's were a leftover from the example I copied, probably add.h.

Yeah, that was my guess.  Obviously not a big deal.

* I move a bunch of implementations from macros into the functions.  I know you thought that the x86 ones required an immediate-mode parameter, but even if that were true the WASM ones don't, there is no need to make them macros for defensive purposes.  This actually revealed a couple of minor bugs regarding types because now the input has to be compatible with int, which is then cast to unsigned char/short/int/long long in the function.

I still maintain that these require an immediate-mode parameter: the IIG even gives the assembler instruction which should be generated. While current compilers seem to work around this limitation, there is absolutely nothing preventing a future compiler from conforming strictly with the IIG. So I don't really agree, but it is your call.

I kind of agree with you in the abstract, but there are simply too many programs in the wild that depend on this behavior.  If some future compiler actually requires them to be an ICE we can work around it for them by adding a macro which is only exposed to that compiler, but I really don't think that's likely to happen.
Meanwhile, there are real benefits (mostly from type safety) to having the implementations in functions, and it's a lot more readable.  IMHO the trade-off is more than worth it.

Sorry for the bugs :-(

Not at all.  None of the stuff I mentioned were really bugs anyways.

I don't think that there will be a "main" header for shr. AFAICS shr is already complete as all shr instructions are _n.

Oh, right.  IIRC for shr (not shr_n) you're supposed to use shl with a negative value.  It's such an odd thing for NEON to be missing; most of their APIs are so complete (especially compared to Intel's).
Anyways, I realized after I wrote this that if peple thing there are too many headers they can always just use the amalgamated one instead, so I don't think this is an issue anyways.

* I move a bunch of implementations from macros into the functions.  I know you thought that the x86 ones required an immediate-mode parameter, but even if that were true the WASM ones don't, there is no need to make them macros for defensive purposes.  This actually revealed a couple of minor bugs regarding types because now the input has to be compatible with int, which is then cast to unsigned char/short/int/long long in the function.

I still maintain that these require an immediate-mode parameter: the IIG even gives the assembler instruction which should be generated. While current compilers seem to work around this limitation, there is absolutely nothing preventing a future compiler from conforming strictly with the IIG. So I don't really agree, but it is your call.

I kind of agree with you in the abstract, but there are simply too many programs in the wild that depend on this behavior. If some future compiler actually requires them to be an ICE we can work around it for them by adding a macro which is only exposed to that compiler, but I really don't think that's likely to happen.

I think we must agree to disagree again on this :-(
You are relying on undocumented behaviour and I don't like that.
(Sorry I can't write behavior: I'm English ;-) )

Meanwhile, there are real benefits (mostly from type safety) to having the implementations in functions, and it's a lot more readable. IMHO the trade-off is more than worth it.

Maybe you are almost starting to come around to my view that, apart from extensions to the language, if should be preferred to #ifdef inside functions as it enables the compiler to compile-test more code paths ;-)

I don't think that there will be a "main" header for shr. AFAICS shr is already complete as all shr instructions are _n.

Oh, right. IIRC for shr (not shr_n) you're supposed to use shl with a negative value. It's such an odd thing for NEON to be missing; most of their APIs are so complete (especially compared to Intel's).

Yes, exactly shr doesn't exist and you have to use shl with a negative value. But I don't like the l in the mnemonic. You can't even say it's l for logical, not left, because there are also arithmetic shifts.


No Python 3 supported

I'm getting the following error on Python 3 (while the setup.py file makes no claims about supported Python versions):
  File "/Users/dinu/miniconda3/lib/python3.5/site-packages/flask_permissions/models.py", line 112, in __init__
    if roles and isinstance(roles, basestring):
NameError: name 'basestring' is not defined

Is this still being developed?

I don't work much in Python these days, and I have not worked on flask-permissions in a long while. It does not have support for Python 3, but I'd gladly take a look at a pull request if you want to submit one.

Well, I'm using this workaround in models.py:
        if roles:
            import sys
            v = sys.version_info.major
            if v == 2 and isinstance(roles, basestring):
                roles = [roles]
            elif v == 3 and isinstance(roles, str):
                roles = [roles]

Please close this issue, fixed in commit 3481373.


Reinstating badges on README for 3rd party tools

Now that SparkTC/spark-bench is enrolled in TravisCI and CodeCov, it's
time to bring back the badges and point them at the appropriate links.

Codecov Report

Merging #64 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master      #64   +/-   ##
=======================================
  Coverage   88.02%   88.02%
=======================================
  Files          26       26
  Lines         551      551
  Branches       28       28
=======================================
  Hits          485      485
  Misses         66       66

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9dc4810...b1cbe1e. Read the comment docs.


Typo? train.py

Hello, I might be horribly mistaken but I also might have seen a "typo".
In train.py first docstring, it says "Train a diffusion model on images."
Is it incorrect?

It's a typo. The code is inherited from image diffusion models.


fix Makefile

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.03%) to 72.139%

Totals

Change from base Build<PHONE_NUMBER>:
0.03%

Covered Lines:
4596

Relevant Lines:
6371

💛  - Coveralls


Missing module?

Shouldn't there be a module called 'utils/shortcut.lua'  included as well? Thanks!

When trying to train an encoder decoder model with attention by calling the script experiments/wikilarge/encdeca/train.sh, I get the error message: "module 'shortcut' not found:No LuaRocks module found for shortcut", although this file does exists in utils/shortcut.lua. Do I need to compile anything in advance such that require('shortcut') and other modules will work?


About stepsArray

I want to change some values to words ( Can't list all the values cause I want to slider step:0.01 )
such as '1' to 'very poor' , '3' to 'fair'. balabala
but I still want to set step: 0.01,precision: 10
so that I can slider to '1.03', '1.05' and so on.
How should I do?
https://jsfiddle.net/170bymjp/2/

If you want to directly replace the value then use the translate function.Otherwise, if you want to display the text as legend below the slider, then use the getLegend option.
For the 2 options, you don't need to use stepsArray

One more question. If I want to show the ticks which I change text only. (show you the picture)
Any settings?
Anyway I can change your code to realize this. thank you very much.

The showTicks can take an integer value to display ticks at intermediate positions.

yes. it works. I like this slider very much

:+1:


Added Two Sum IV - Input is a BST using CPP

Signed-off-by: Umesh Chandra<EMAIL_ADDRESS>

@all-contributors please add @MrCuber for his code


Add default timeout to qsharp._execute

Currently, by default there is currently no timeout for call to the IQsharp kernel from the qsharp package. In general this has and should not been a problem as most calls will return immediately. However, some of the tests run during conda validation are causing some deadlock and the call never returns.
Changing the default timeout to be 2 min for most operations. Leaving None only for simulate and estimate as those can be long-lived operations.

I think this looks good, thanks for adding this! Just had the one question about how to handle turning off timeouts in simulate and friends. Thanks!

/azp run


Bbox documentation

First stab at handling GeoJSON BBox & Google's Bounds input.
@ebreton Might be worth having a helper method to handle BBox across all providers that support BBox.

Hi @DenisCarriere ,
I followed on your first stab 👍 and your suggestion to add a helper for bbox: it's a new class in location.py with a factory (I have added the appropriate tests also, which give some examples too)
-> I have adapted consequently the three providers that offer a proximity query:

geonames
google place
mapbox

The new BBox class could surely be used in the other cases where a bbox is implied, but I prefer that you have a look at it beforehands, and see for yourself if that worth adapting further.
The doc is also reviewed with all this (and also the api.py docstrings). As usual: feel free to enrich / correct.

What about adding some columns in the the README.

👍 That sounds like a good plan

actually... done ;)

Oh perfect! 👍


Updated FindRating function

Preview Link

Click to demo updates
Description

Motivation and Context

This PR fixes the FindRating function that did not properly sort the low, medium, and high values.
How has this been tested?

Screenshots (if appropriate):

Types of changes

Refactor (code improvement that doesn't affect UI)

Awesome,  thank you!


Drafting: Living Atlases

A page linked from the menu describing what The Atlas project (by ALA) is.

Per @timrobertson100: we will create a “programme” entry for the Atlases, using a content model that serves as an umbrella entity over projects (the same that currently serves BID, BIFA and the CEP sections).
We'll create a project for each partner involved (GBIF.es, GBIF France, NBN etc), and create entries for the events that have happened, link out to their installations etc.  Once the basics were in place we’d work with the groups directly to complete a summary of their status, and ideally their roadmap. This approach  will help demonstrate the capacity enhancement and knowledge exchange underway and this is directly addressing what GBIF France requested.
It would appear in search, and initially under our new menu structure linked under the tools section we’ve identified as needed.  We might revise this as content fills out and it might end up under the Inside GBIF section, or both – we’re in these kind of discussions with the comms team now so too early to decide definitively.

Much can probably be reused from here on the summary info, and these resources should probably be linked for download (with attribution to the ALA for the Infrastructure, and GBIF Belgium/Spain/France + maybe more? for the KTD)
KTDfinal_en (1).pdf
ALA Infrastructure Implementation - overview (24 6 16) - final (1).pdf

Link existing CE support project, Encounter Bay
News item on GBIF.es launch
Very early (2011) training press
Do we need to do any clean-up to tighten materials on key earlier projects and events, like

ALA-INBio mentoring project, 2012 news item or event (check with mraymond on project description and/or report?)
ALA Workshop, July 2014
GBIF/ALA workshop, Paris (May 2015)
CoopBioPlat workshop, Madrid: Oct 2015 news item or event](http://www.gbif.org/event/82290); also linked to Encounter Bay (see above)
BGBM workshop to investigate use of ALA components for BGBM projects (Nov 2015)
GBIF France workshop on spatial components (Feb 2016)

Note also connection to Symbiota (Northern Arizona University), which received a NSF grant to implement/integrate ALA Spatial portal components into their software
Also: original Brazilian portal

Is the overarching NBN Atlas one project or four/five? I think I know this, but I'm just asking...

Tricky one to answer actually... Architecturally speaking it is 1 backend system and data management practice, but will be presented as ALS, ALW, ALE to the user.  I don't know if there will be an ALUK or not.

Agreed. This is what I wanted to dig up, make sure I didn't dream it or something stupid like that: NBN's take on it all
NBN Atlas – the name
The new Atlases will all have the pre-fix “NBN” to ensure consistency of branding and strengthening of the NBN name.  Therefore, the various portals will be known as follows:

NBN Atlas – the overarching Atlas and underlying database
NBN Atlas England
NBN Atlas Northern Ireland
NBN Atlas Scotland
NBN Atlas Wales

The project on GBIF should thus be "NBN Atlas", with links to each of the four nations' sites, no?

I'd agree that was best.  1 project, and 4 links and description of their
set up, roadmap, they can describe their approach, funding etc etc.
+1
On Fri, Oct 14, 2016 at 9:41 AM, kcopas<EMAIL_ADDRESS>wrote:

Agreed. This is what I wanted to dig up, make sure I didn't dream it or
something stupid like that: NBN's take on it all
NBN Atlas – the name https://nbn.org.uk/news/nbn-atlas-brief-update
The new Atlases will all have the pre-fix “NBN” to ensure consistency of
branding and strengthening of the NBN name. Therefore, the various portals
will be known as follows:

NBN Atlas – the overarching Atlas and underlying database
NBN Atlas England
NBN Atlas Northern Ireland
NBN Atlas Scotland
NBN Atlas Wales

The project on GBIF should thus be "NBN Atlas", with links to each of the
four nations' sites, no?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/gbif/portal16/issues/87#issuecomment-253731291, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAOepem38EBnSKXq9xgVK2DOryE3LEmfks5qzzJFgaJpZM4KUjng
.

Can I ask: is "Atlases" the label? My gut instinct is that this will be poorly understood unless you already know what you're looking for, and that something like 'Living Atlases' might be preferable.
That approach would also make the ALA homage a bit more explicit and (hopefully) clearer. For thought and later discussion...

It's not settled, and the ALA folk should be involved in the decision.  I
am sure they are open to suggestions.
On Fri, Oct 14, 2016 at 10:50 AM, kcopas<EMAIL_ADDRESS>wrote:

Can I ask: is "Atlases" the label? My gut instinct is that this will be
poorly understood unless you already know what you're looking for, and that
something like 'Living Atlases' might be preferable.
That approach would also make the ALA homage a bit more explicit and
(hopefully) clearer. For thought and later discussion...
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/gbif/portal16/issues/87#issuecomment-253744259, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAOepU0R8FCC0u--NKlDWdaW2EGAfJLcks5qz0I8gaJpZM4KUjng
.

John LaSalle gave 'Living Atlases' the thumbs-up at GB23. Do we need to check in with anyone else—@davemartin maybe?
Will hold off updating the title of this issue until resolved…

DJM says: "if John’s happy, I’m happy"

+1

Hi – not sure why I’ve received this message, guess someone has got me confused with another Dave Martin
This is Dave Martin in the Isle of Man – Dave<EMAIL_ADDRESS>From: kcopas<EMAIL_ADDRESS>Sent: 01 November 2016 14:43
To: gbif/portal16
Cc: DaveMartin; Mention
Subject: Re: [gbif/portal16] Drafting: Atlas (by the ALA) (#87)
John LaSalle gave 'Living Atlases' the thumbs-up at GB23. Do we need to check in with anyone else—@davemartin https://github.com/davemartin  maybe?
Will hold off updating the title of this issue until resolved…
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub https://github.com/gbif/portal16/issues/87#issuecomment-257584229 , or mute the thread https://github.com/notifications/unsubscribe-auth/ADGQ6L8Fc5YFAx33iwlw9Bq1fNgqNaZzks5q50_igaJpZM4KUjng .  https://github.com/notifications/beacon/ADGQ6MRcjRhpfNB1PQKrHb2TUq7JsxOLks5q50_igaJpZM4KUjng.gif

Just to say thanks @kcopas @timrobertson100. This is all sounding really great :)

Hi – not sure why I’ve received this message, guess someone has got me confused with another Dave Martin
This is Dave Martin in the Isle of Man – Dave<EMAIL_ADDRESS>From: Dave Martin<EMAIL_ADDRESS>Sent: 01 November 2016 20:29
To: gbif/portal16
Cc: DaveMartin; Mention
Subject: Re: [gbif/portal16] Drafting: Living Atlases (#87)
Just to say thanks @kcopas https://github.com/kcopas  @timrobertson100 https://github.com/timrobertson100 . This is all sounding really great :)
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub https://github.com/gbif/portal16/issues/87#issuecomment-257685601 , or mute the thread https://github.com/notifications/unsubscribe-auth/ADGQ6OPJ0JigYU0qRHu_JmMzzKzzSUK1ks5q56EKgaJpZM4KUjng .  https://github.com/notifications/beacon/ADGQ6OvYd0DIb0Wl1Tx7_HNG12ZhjO11ks5q56EKgaJpZM4KUjng.gif

Sorry @DaveMartin, indeed the message was intended to @djtfmartin. I changed the original message. Do you want me to delete your answers above? (so you won't receive more notifications).
Sorry about that

Thanks Christian – if you can easily do so, it might be best to delete any mention of my name – also so its clear that the Dave Martin they were intended for didn’t get them!
Thanks again & kind regards / Dave (IOM)
From: Christian Gendreau<EMAIL_ADDRESS>Sent: 01 November 2016 20:34
To: gbif/portal16
Cc: DaveMartin; Mention
Subject: Re: [gbif/portal16] Drafting: Living Atlases (#87)
Sorry @DaveMartin https://github.com/DaveMartin , indeed the message was intended to @djtfmartin https://github.com/djtfmartin . I changed the original message. Do you want me to delete your answers above? (so you won't receive more notifications).
Sorry about that
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub https://github.com/gbif/portal16/issues/87#issuecomment-257686967 , or mute the thread https://github.com/notifications/unsubscribe-auth/ADGQ6OSMz4cT7M6olETnhe52c7aN3Ddeks5q56I1gaJpZM4KUjng .  https://github.com/notifications/beacon/ADGQ6Hiq2qzqrVnTI9iymaTOJ9AztFKeks5q56I1gaJpZM4KUjng.gif

Deep background, discovered by chance in the comms section of Box.com

Australia Government Awards $30 million to Develop System to Manage Australia’s Biological DiversityDataCanberra, Australia, 25 May 2009.
The Australian government announced on 12 May the award of $30 million for the Atlas of Living Australia (ALA) for work during 2009-2011.
Commonwealth Scientific and Industrial Research Organisation(CSIRO) Group Executive for Agribusiness, Dr Joanne Daly, said, “the funding will help develop the Atlas of Living Australia into a world-class research tool.”  Hosted by CSIRO and in collaboration with partners in government, museums and the universities, the program is developing a biodiversity data management system which will link Australia’s biological knowledge with its scientific and agricultural reference collections and other custodians of biological information.
“To have integrated information on all Australian species, including data on specimens held by Australia’s natural history collections and data from field observations of living organisms is a very valuable research tool,” Dr. Daly said.  “The funds will allow CSIRO and our partners to support the management and integration of biological data from all areas of research from molecular to ecological,” Daly continued.ALA was established by the Australian government in 2007 as a collaboration between Australian government agencies, museums and universities with government funding totaling $8.2 million for the period 2007-2011.
The goal of the project is to build web infrastructure to link information from institutions and agencies throughout Australia on all species found inthe continentand its surrounding waters. This will be a foundational resource for research, policy and education.In particular the ALA will allow researchers to understand the variation in biodiversity across the landscape and to use this information for the benefit of conservation and agriculture.
The ALA will also support Australia's biosecurity response to new pests, weeds and diseases, particularly as species distributions alter under climate change.The ALA will use web technologies to allow users and experts to add to existing information and to improve the quality of the data and tools.The new $30 million ensures that ALA can deliver a comprehensive set of tools for managing distribution data, descriptions, images, literature and molecular data for any species.  These systems will assist Australian researchersin responding rapidly to new discoveries and tobase planning and policy decisions on the fullest available scientificdata and will feed directly into international networks and projects such as GBIF and EOL

The programme/project complex for Living Atlases is stubbed out and (I hope) ready to share provisionally with the actual participants (Participants) for refinement, once we can get it to display on demo.

@kcopas - this one is done, right?


How to fix one end of the Brush in D3.js?

I have a Slider created using D3 and want to fix one end of the slider to Starting of the Brush extent whereas want the other end to be draggable so user can change the selection.
I am able to disable the entire dragging but not sure how to keep dragging enabled but fix one end of the brush.
Any help is highly appreciated.
Thanks :)

Sure, you could use the same technique as in the brush snapping example:
https://observablehq.com/@d3/brush-snapping
Instead of rounding, though, you’d fix the start of the brush’s selection.
Sorry for the delayed response!


Fix for flaky keyservers #11601

Description
See title. Fixes #11601
Checklist
Essentials
Please feel free to remove inapplicable items for your PR.

[x] The PR title starts with [MXNET-$JIRA_ID], where $JIRA_ID refers to the relevant JIRA issue created (except PRs with tiny changes)
[x] Changes are complete (i.e. I finished coding on this PR)
[x] All changes have test coverage:
Unit tests are added for small changes to verify correctness (e.g. adding a new operator)
Nightly tests are added for complicated/long-running ones (e.g. changing distributed kvstore)
Build tests will be added for build configuration changes (e.g. adding a new build option with NCCL)
[x] Code is well-documented:
For user-facing API changes, API doc string has been updated.
For new C++ functions in header files, their functionalities and arguments are documented.
For new examples, README.md is added to explain the what the example does, the source of the dataset, expected performance on test set and reference to the original paper if applicable
Check the API doc at http://mxnet-ci-doc.s3-accelerate.dualstack.amazonaws.com/PR-$PR_ID/$BUILD_ID/index.html
[x] To the my best knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change

Thanks for addressing this Pedro.  This will improve reliability quite a bit.

@szha

@marcoabreu please merge

Thank you Marco. We missed you.


Use initialize and generate getDependentDialects for passes from IREE

Extending work done in #1919 and #1966.

Can you rebase, I think CHLO->StableHLO now uses the declarative dependencies


Adding custom_gradient example

Addresses #26270.
As mentioned by @dynamicwebpaige in #26270 (Comment).
cc @mihaimaruseac, @lintian06. @alextp

@MarkDaoust , @wolffg , @yashk2810 - Any updates on this issue?

Hi @ashutosh1919,
If we put this example here nobody will ever find it.
The original issue you link to is old (>2 years).
We've added better docs for this sort of thing in these two docs:

https://www.tensorflow.org/guide/autodiff
https://www.tensorflow.org/guide/advanced_autodiff#custom_gradients

That second link includes a custom gradient example. Read that section. If you feel it's still missing something, please send a PR against this file: https://github.com/tensorflow/docs/blob/master/site/en/guide/advanced_autodiff.ipynb (in the tensorflow/docs repository)


Unexpected errors when running in a React Native project

I have a pretty standard project using React Native. It. has some cocoapods installed or the iOS version.
When I run npx tsconfig.js --once I get heaps (3829 to be exact) of errors like the one below (I included only the last one as all of them are basically the same)
error:   Invalid or unexpected token
error:   Error while processing /path/to/rn/project/ios/Pods/Analytics/Segment/Internal/UIViewController+SEGScreen.h
error:   Unexpected identifier
info:    Processing complete, with 3829 errors, see above for details
npm ERR! code 1
npm ERR! path /path/to/rn/project
npm ERR! command failed
npm ERR! command sh -c tsconfig.js "--once"

It seems that tsconfig.js tried to process all *.h files unnecessarily. When I enabled the debug logs, I saw:
debug:   Looking for files matching [
        /path/to/rn/project/**/tsconfig.js
    ]

None of the *.h files match that pattern so I guess it must be some sort of a bug.
I worked around it by ignoring the ios folder and running npx tsconfig.js --once -- ios

Hello @jmarek-sky
Thank you for the bug report! Can you provide a simple reproduction for debugging? I'd like to add that as a test case.
A quick look at the code there's no obvious source for such an error. Except if chokidar did this wrong, but I doubt that and I found no similar issue in their repository.
Additional feedback would be appreciated, thanks 🙂

Hi @c-vetter
Sorry for a very delayed response, but I missed the email notification from GitHub among heaps of other emails.
I encountered the same issue in one of my pet projects which is open sourced, so I can share the steps to reproduce with you:

git clone git@github.com:jar0s/reactNative-babel-wla.git
cd reactNative-babel-wla
yarn install
cd ios_template && pod install && cd ..
npx tsconfig.js --once
Observe errors in the console output, last few:

error:   Error while processing /path/to/reactNative-babel-wla/ios_template/Pods/CocoaAsyncSocket/Source/GCD/GCDAsyncSocket.h
error:   Unexpected identifier
error:   Error while processing /path/to/reactNative-babel-wla/ios_template/Pods/CocoaAsyncSocket/Source/GCD/GCDAsyncUdpSocket.h
error:   Unexpected identifier
info:    Processing complete, with 3702 errors, see above for details


Hue Lights

The problem
After configuring the integration, I have had my Hue lights show up in the Vivint panel.  I can remove the lights by contacting support to enable installer mode, then pairing/removing the hue bridge, but the lights show up between 1-24 hours after the bridge is added/removed.  Is the integration capable of doing this?  If so, is there a way to prevent devices from zha from being seen by the panel?
What version of Home Assistant Core are you running?
core-2024.5.5
What was the last working version of Home Assistant Core?
No response
Anything in the logs that might be useful?
No response
Additional information
The account I am using to integrate is dedicated and is not configured with admin rights.

The integration is not able to do this.

Thank you, Nate.  I will continue to troubleshoot with Vivint.


Merge with pypa/distutils@b65aa40

Convert test_cmd to pytest
Convert BasePyPIRCCommandTestCase to pytest
Convert RegisterTestCase to pytest
⚫ Fade to black.
Use jaraco.path to generate a tree.
Port sdist tests to pytest
Convert TestUpload to pytest
Convert TestUpload to parametrized test. Remove dependence on unittest for PyPIRC tests.
Implement HTTP 400 error as a pytest.param to avoid collection error. Ref pytest-dev/pytest#10184.
Convert core tests to pytest
⚫ Fade to black.
Convert TestVersion to pytest.
Convert TestConfig to pytest
Convert TestCygwinCCompiler to pytest
Convert TestDirUtil to pytest
Convert TestFileUtil to pytest
Convert TestSysconfig to pytest
⚫ Fade to black.
👹 Feed the hobgoblins (delint).
Remove patching of uname.
Convert TestUtil to pytest
Convert TestUnixCCompiler to pytest
Prefer pytest for skip
Convert more tests to pytest
Prefer pytest for skip
Prefer pytest for skip
Convert TestBuild to pytest
Prefer pytest for skip
Prefer pytest for skip
Remove unreachable code
Copy xxmodule.c from Python 3.11 and 3.8, restoring tests for build_ext.
Exclude Python 3.11 on macOS due to lack of wheels. Ref pypa/distutils#165.
Mark test as xfail for now. Ref pypa/distutils#166.
Use pathlib to read the text
⚫ Fade to black.
Convert TestInstall to pytest
Only xfail on Windows
Allow overriding toxworkdir with an env var.
Ensure sys.version is restored in test_cygwinccompiler. Fixes #166.
Run test_xx in process, utilizing import_helper On Windows, move the extension module to another temporary directory.
Redirect extension module to a directory that's not deleted on Windows.
Include cygwin
Prefer pytest in test_build_py
Prefer pytest in test_check
Refactor imports around docutils.
Replace addCleanup with monkeypatch.
Enable tests requiring docutils.
Fix broken tests around docutils.
Convert PendingDeprecationWarnings to DeprecationWarnings.
Ignore unactionable warnings in docutils.
Prefer pytest for skip
Prefer pytest for skip
Prefer pytest for skip
Prefer pytest for skip
Prefer pytest for skip
Convert TestLog to pytest.
Prefer pytest for skip
Consolidate tests
Prefer pytest for skip
Prefer pytest for skip
Convert unconditional skip to conditional skip. Mark test as xfail because it's failing.
Prefer pytest for skip
⚫ Fade to black.
Convert unix_compat to pytest skips
Consistently import unittest.mock.
Prefer pytest for skip
Remove unnecessary comment.
⚫ Fade to black.
👹 Feed the hobgoblins (delint).
Prefer tabs
Add pytest-flake8 and pytest-black and pytest-cov to test lint and style and coverage
👹 Feed the hobgoblins (delint).
Add test capturing failed expectation.
Add compatibility for Python 3.7
Windows is sensitive even on Python 3.10
Also test library dirs
Extract fixture for c_file
Generate a C file that imports Python.h and something platform specific.
Ensure Python include directory is configured.
Extend the test to compile a second time after setting include dirs again.
Ensure Windows SDK directories are not cleared when caller specifies include/library dirs
Remove stray colon
Fixup bad super() call
Use CCompiler._fix_compile_args to fix args to compile()
⚫ Fade to black.
Allow compiler classes to supply include and library dirs at the class level.
Disallow repeat calls to .initialize in one place.

Summary of changes

Closes
Pull Request Checklist

[x] Changes have tests
[x] News fragment added in changelog.d/.
(See documentation for details)

@jaraco This PR pulls includes 9f9a3e57643cb49796c1b08b5b5afb2826ecd7f6, which I think introduces a bug into setuptools 63.4.3.  see #3591  I don't think I understand the workflow behind this PR.  Where should a fix be made?


Doesn't seem to work

On anything else beside your example video
try this https://m.youtube.com/watch?v=-CmadmM5cOk
Signature extraction failed: Traceback (most recent call last):
File "/var/mobile/Containers/Data/Application/3A8E99FD-AFD6-4DEA-9BDB-7B34E8C8626B/Documents/site-packages/youtube_dl/extractor/youtube.py", line 634, in _decrypt_signature
video_id, player_url, s
File "/var/mobile/Containers/Data/Application/3A8E99FD-AFD6-4DEA-9BDB-7B34E8C8626B/Documents/site-packages/youtube_dl/extractor/youtube.py", line 548, in _extract_signature_function
res = self._parse_sig_js(code)
File "/var/mobile/Containers/Data/Application/3A8E99FD-AFD6-4DEA-9BDB-7B34E8C8626B/Documents/site-packages/youtube_dl/extractor/youtube.py", line 612, in _parse_sig_js
initial_function = jsi.extract_function(funcname)
File "/var/mobile/Containers/Data/Application/3A8E99FD-AFD6-4DEA-9BDB-7B34E8C8626B/Documents/site-packages/youtube_dl/jsinterp.py", line 244, in extract_function
raise ExtractorError('Could not find JS function %r' % funcname)
ExtractorError: Could not find JS function u'gG'; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output

If you have trouble with a specific video, try to use the non-mobile url (e.g. http://youtu.be/-CmadmM5cOk or https://www.youtube.com/watch?v=-CmadmM5cOk )
See also https://github.com/rg3/youtube-dl/issues/2858

Alternative urls throws exceptions as well. Just as other popular videos on youtube.
I am just pointing out that this code isn't working or throws exceptions for several, popular youtube videos. So I wonder if you have tested it a bit  before 'commit'. This is github not facebook.
As for 'mobile url' - you are attempting to make a fork for Pythonista which seems to be a mobile app.
Sure, going through the code and making a working fork myself won't be that difficult.
[0;33mWARNING: Writing cache to u'/private/var/mobile/Containers/Data/Application/3A8E99FD-AFD6-4DEA-9BDB-7B34E8C8626B/.cache/youtube-dl/youtube-sigfuncs/js_en_US-vflT9SJ6t_40.43.json' failed: Traceback (most recent call last):
File "/var/mobile/Containers/Data/Application/3A8E99FD-AFD6-4DEA-9BDB-7B34E8C8626B/Documents/site-packages/youtube_dl/cache.py", line 46, in store
os.makedirs(os.path.dirname(fn))
File "/private/var/mobile/Containers/Bundle/Application/0F009021-6E5A-4727-A692-5E7231941A67/Pythonista.app/pylib/os.py", line 150, in makedirs
makedirs(head, mode)
File "/private/var/mobile/Containers/Bundle/Application/0F009021-6E5A-4727-A692-5E7231941A67/Pythonista.app/pylib/os.py", line 150, in makedirs
makedirs(head, mode)
File "/private/var/mobile/Containers/Bundle/Application/0F009021-6E5A-4727-A692-5E7231941A67/Pythonista.app/pylib/os.py", line 157, in makedirs
mkdir(name, mode)
OSError: [Errno 1] Operation not permitted: '/private/var/mobile/Containers/Data/Application/3A8E99FD-AFD6-4DEA-9BDB-7B34E8C8626B/.cache'

The video that you posted has been tested (and works using the alternate urls), as has many other videos. If there's a specific bug that seems related specifically to this fork please post it here. Otherwise this seems like an issue related to the upstream branch.
In general, the youtube-dl library is very flimsy because much of it relies on screen scraping. I apologize that I haven't updated this as frequently as I would like and is needed. As an alternative, you can try the youtubedl-downloader script by @shaun-h
The scripts that were provided in the README were only examples - you can trivially add a mobile url "converter" to your own script.


Adapter not working

NewRelic Istio adatper not working, I installed it on our k8s cluster 10.5 version and istio 1.0.5
I have followed readme.md,
1 Copied k8s deployment files from /kube/ folder
2deployed on istio-system namespace with our new relic api and accountid
3 adapter is running , no issues
4. When icheck the NewRelic -> insights  -> Events, I could not see istio metrics under custom events
Did i miss any thing, Please help me out

@dgurindapalli same issue here. Beyond your steps, I also applied an egress rule to ensure traffic wasn't blocked to insights-collector.newrelic.com and applied a version of the newrelic/testdata/sample_operator_cfg.yaml with the K8s service DNS.
@wentao-zh - I'm not seeing any adapters deployed after following the above steps. Perhaps the documentation is incomplete for a Kubernetes/Istio setup?

Please use the Istio adapter built/support by New Relic instead: https://github.com/newrelic/newrelic-istio-adapter


Faster bulk insertion

I'm using rexie in a project to insert a large number of entries into a database, on the order of 300k. As this takes its sweet time, especially on mobile devices, I'm looking into ways to speed it up. I've seen that you can get a significant speedup by not subscribing to success events in add or put calls.
I have done a quick and dirty test on the potential speedup in rexie, and it may be as large as 50% or more.

Benchmark implementation outline
I'm not sure how to go about benchmarking rust in a browser properly. I have simply added 2 simple tests and verified the correct data was present afterwards.
// Add to tests/web.rs

async fn setup(test_name: &str) -> Rexie {
  Rexie::delete(test_name).await.unwrap();

  let rexie = Rexie::builder(test_name)
      .version(1)
      .add_object_store(ObjectStore::new("nums").key_path("id").auto_increment(true))
      .build()
      .await
      .unwrap();

  rexie
}

#[derive(Serialize)]
struct EmptyStruct {}

const ITERATIONS: u32 = 10_000;

#[wasm_bindgen_test]
async fn bench_simple() {
  let test_name = "bench_simple";
  let rexie = setup(test_name).await;

  let transaction = rexie
      .transaction(&["nums"], TransactionMode::ReadWrite)
      .unwrap();

  let nums = transaction.store("nums").unwrap();

  let obj = EmptyStruct {};
  let obj = serde_wasm_bindgen::to_value(&obj).unwrap();

  web_sys::console::time_with_label(test_name);

  for _ in 0..ITERATIONS {
      nums.add(&obj, None).await.unwrap();
  }

  transaction.commit().await.unwrap();

  web_sys::console::time_end_with_label(test_name);
}

#[wasm_bindgen_test]
async fn bench_bulk() {
  let test_name = "bench_bulk";
  let rexie = setup(test_name).await;

  let transaction = rexie
      .transaction(&["nums"], TransactionMode::ReadWrite)
      .unwrap();

  let nums = transaction.store("nums").unwrap();

  let obj = EmptyStruct {};
  let obj = serde_wasm_bindgen::to_value(&obj).unwrap();

  web_sys::console::time_with_label(test_name);

  for _ in 0..ITERATIONS {
      nums.add_bulk(&obj, None).await.unwrap();
  }

  transaction.commit().await.unwrap();

  web_sys::console::time_end_with_label(test_name);
}

// src/transaction/store.rs impl Store

    pub async fn add_bulk(&self, value: &JsValue, key: Option<&JsValue>) -> Result<JsValue> {
        let request = match key {
            Some(key) => self.idb_store.add_with_key(value, key),
            None => self.idb_store.add(value),
        }
        .map_err(Error::IndexedDbRequestError)?;

        wait_request_bulk(request, Error::IndexedDbRequestError).await
    }

// src/request.rs

pub async fn wait_request_bulk<R: DbRequest + Unpin>(
    request: R,
    map_err: fn(JsValue) -> Error,
) -> crate::Result<JsValue> {
    RequestFuture::new_bulk(request, map_err).await
}

impl<R> RequestFuture<R>
where
    R: DbRequest + Unpin,
{
    pub fn new_bulk(request: R, map_err: fn(JsValue) -> Error) -> Self {
        let (sender, receiver) = mpsc::unbounded_channel();

        sender.send(Ok(JsValue::null())).unwrap();

        let success_closure = get_success_closure(sender.clone());
        let error_closure = get_error_closure(sender);

        // request.on_success(Some(get_callback(&success_closure)));
        request.on_error(Some(get_callback(&error_closure)));

        Self {
            _inner: request,
            _success_closure: success_closure,
            _error_closure: error_closure,
            receiver,
            map_err,
        }
    }
}

Numbers
Chrome:
bench_bulk: 1153.651123046875 ms
bench_simple: 2531.212158203125 ms

Firefox:
bench_bulk: 443.04ms
bench_simple: 1957.06ms

The implementation outlined above is obviously hacky and dirty, and I'm not sure what the best API for it would be. It would be something I'd quite like to see, however.

Hi. Another option is to not await the future returned by add in a loop. Instead, storing futures in a Vec use something like join_all to await for all the futures at once. Will that help?

It's a good idea I hadn't considered. Using join_all from the futures crate like this:
futures::future::join_all(
    (0..ITERATIONS)
        .map(|_| nums.add(&obj, None))
        .collect::<Vec<_>>(),
)
.await
.into_iter()
.collect::<Result<Vec<_>>>()
.unwrap();

I get these timings on chrome
bench_bulk:<PHONE_NUMBER>21875 ms
bench_join_all:<PHONE_NUMBER>375 ms
bench_simple: 1650.171142578125 ms

And these on firefox:
bench_bulk: 275.3ms
bench_join_all: 656.92ms
bench_simple: 991.52ms

Timings are better than the previous numbers because I realised I wasn't running in release mode.
So, using join_all is much better than awaiting each element separately, but it is, for this test, still significantly slower than skipping those awaits altogether.

They way I understand this is that you're not waiting for value to be inserted in the database before yielding. You can have the same effect in current code by not .awaiting the future returned by add. It will immediately return without waiting for success. Intuitively, immediately returning can be more faster than returning null from async call.
You'll get a compiler warning by not awaiting the future but you can ignore that because browser runtime automatically schedules the task (rust runtime needs future to be polled in order to move forward as opposed to browser runtimes).

Hmm, I've tried to do it like this, and it finishes pretty much instantly and leaves the database empty.
 (0..ITERATIONS).for_each(|_| {
     nums.add(&obj, None);
 });

I would also like to see this improvement implemented, preferably with a function that takes an iterator. Something like this:
pub async fn add_iter<I>(iter: I) -> Result<JsValue>
where
    I: IntoIterator<Item = (JsValue, Option<JsValue>)>,
{
    // Loop over the elements in the iterator
}

Then it could iterate over the provided data and attach an on_success callback only to the operation on the last element produced by the iterator. It would almost be like calling add_bulk on all the data except for the last one that would use add for that one on_success callback to trigger a resolution to the Future.
You can't really undo the effect of setting a bunch of on_success callbacks, so the best approach is to not set them in the first place.

@FlippingBinary and @ShaddyDC. I'll look into it over the weekend. I anyway think that the API needs some restructuring for more ease-of-use.

@devashishdxt I would love to be a part of that discussion, if you're open to it. This crate has quickly become important to a research project I'm working on. I don't want to write a PR without knowing in advance you'd be receptive to the approach, so I'll wait to hear more about your thoughts on restructuring.

@FlippingBinary I've experimented with a proc macro based API in deli. I'm still not entirely convinced with the proc macro based API as I think it is too opinionated. It'd be great if you can take a look and provide feedback. For experimentation, you are most welcome to create PR on either rexie or deli. Thanks for your interest!

Unfortunately, deli would have the same performance issue on bulk inserts because it uses idb, which sets the on_success callback on every request. It looks like you own all the IndexedDB crates, lol. I'll have to put some thought into which one I'd rather contribute to. I know it wouldn't be deli because I have to prioritize the performance boost described above, but I do like the usage of macros for describing the models. Have you considered integrating some features of deli with rexie? I'm not sure if it's your intention, but rexie seems to be most appropriate for a "batteries included" type of crate.

Solving the bulk insert part is not that difficult. You can collect all the futures in a vec and join them in one go. I think that'll be much faster. I can try adding a function which internally does this.

That's already what I do in my project. I hope I can get better performance out of it! According to the StackOverflow answer, the JS runtime causes the performance problem when the on_success callback is set on a large number of add/put operations. Both rexie and idb set it on every request. It's how the Future is resolved. Instead of setting it on every request, it needs to set it on only the last request in a series. Then it should return an error if any of the error handlers are triggered, or return success if the only success handler fires.
You can see what I'm describing in Dexie's mutate function: https://github.com/dexie/Dexie.js/blob/cb1793a40f9a7cf02010f0cd134831e5523e007f/src/dbcore/dbcore-indexeddb.ts#L115

I've made some changes in latest version of idb crate which should enable faster bulk insertion. I'll soon port those changes to rexie too.

@devashishdxt thank you for your work on these crates. Will Deli get the bulk add performance improvements applied from idb?

@devashishdxt thank you for your work on these crates. Will Deli get the bulk insertion performance improvements applied from idb?

Not yet. After I upgrade the dependencies and release a new version. I'll release a new version of deli soon.

I've created a PR for this #46. Unfortunately, this PR only supports bulk insertion and updation on a single object store. If you want to perform bulk operations across object store, I'd suggest you use idb crate.


U+ Connect Module Info / Captures

The U+ Connect module (GE Part# PBX23W00Y0) provides cloud based interface to select appliances.   Module is in a small plastic case with a single push button and led on front, RJ45 port for connection to appliance, and internal magnets to secure product in rear housing.  It comes with a 12-inch Ethernet cable to connect to appliance.  FCC ID is ZKJ-WCATC001, report is here.
Connected it to my dryer for initial testing.  The Android app to establish the connection is quite buggy, lots of random closes/crashes and not very responsive.  Basic status for dryer settings were passed back to the app.  It did not seem to support turning dryer on remotely.  It had an option and confirmation dialog box to turn dryer off remotely but this function didn't actually work.  Pin 7 had the expected19200 serial traffic.  Here was sample capture of the traffic.
PCB photos:
Top

Bottom1:

Bottom2:

Thanks for the pictures.
I'm seeing only six instances of the start-of-packet byte E2 in your capture. Did you invert the RX pin when capturing over serial? If you have an FTDI USB-serial adapter, it can be done with their FT prog utility:

I was using RealTerm.  The invert option was not checked yesterday.

It's not a software tweak, it's something done in firmware or hardware. The serial data sent by the bus can erroneously be interpreted as "normal" serial but it isn't usable data.

I managed to get an ESP32 hooked up to log the serial data remotely with custom ESPHome UART.  (Don't have to lug laptop to laundry now.)  It is configured with the UART inverted in hardware.  Only the RX is hooked up currently.  Here were a few captures.  Looks like e2 and e3 for packet start and end are coming across.
capture1.txt
capture2.txt
captur3e.txt

I've re-written a large portion of the code. Try it out:
$ virtualenv -p 3.9 venv
$ source venv/bin/activate
$ git clone https://github.com/puddly/casserole
$ pip install -e ./casserole
$ cat capture2.txt | python -m casserole.tools.parser
2021-12-22 20:51:27 ubuntu.local __main__[48214] DEBUG RX: <E2 24 0B BE F0 01 20 12 56 3F E3 E1>
2021-12-22 20:51:27 ubuntu.local __main__[48214] INFO Parsed: <GEAFrame(src=0xBE, dst=0x24, payload=<F0 01 20 12>, ack=True>
2021-12-22 20:51:27 ubuntu.local __main__[48214] INFO Parsed payload: <ERDCommand(command=<ERDCommandID.READ: 0xF0>, erds=[0x2012:None])>
2021-12-22 20:51:27 ubuntu.local __main__[48214] DEBUG RX: <E2 BE 0D 24 F0 01 20 12 01 01 AC 66 E3 E1>
2021-12-22 20:51:27 ubuntu.local __main__[48214] INFO Parsed: <GEAFrame(src=0x24, dst=0xBE, payload=<F0 01 20 12 01 01>, ack=True>
2021-12-22 20:51:27 ubuntu.local __main__[48214] INFO Parsed payload: <ERDCommand(command=<ERDCommandID.READ: 0xF0>, erds=[0x2012:<01>])>
2021-12-22 20:51:27 ubuntu.local __main__[48214] DEBUG RX: <E2 24 08 BE F3 1C 31 E3 E1>
2021-12-22 20:51:27 ubuntu.local __main__[48214] INFO Parsed: <GEAFrame(src=0xBE, dst=0x24, payload=<F3>, ack=True>
2021-12-22 20:51:27 ubuntu.local __main__[48214] WARNING Failed to parse ERD: <F3>
...

The packets are also a little unstructured so I think yours is using the GEA1 protocol, like my washer. There's apparently now a GEA2 and GEA3.
The utility decodes bytes read from lines over stdin, so you can feed it any data as long as it looks something like this (must contain uart_debug and have hex bytes after <<<):
uart_debug <<< BF 0B 24 A0 01 60 03 9B 51 E3

Looks like my washer is GEA3.  Here was a sample capture using two uarts on ESP32 to capture both lines between U+ and the washer.  Uarts configured for 230400 baud but not inverted.
wcap1.txt
The casserole parser has been very useful for decoding.
wcap1.out.txt


Added support for downloading nvdcve-1.1-recent.

The NVD added another file to their data feed called nvd-1.1-recent, which is a feed of recent CVEs (seemingly within the last week). The existing version of NIST Data Mirror does not support any downloads related to nvd-1.1-recent, so this PR aims to remedy this. I've verified locally that running the built .jar will now also download the related .zip, .gz, and .meta files pertinent to nvd-1.1-recent.
(recreated to update some authorship information, super sorry about that!)

Thanks for the PR


add changelog for EKS-A v0.14.6

Issue #, if available:
Description of changes:
Testing (if applicable):
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

/lgtm
/approve

/cherry-pick main


Husky configuration is broken after update to v5

PR #155 brought a Husky major version update (v5) that requires some configuration file changes which were not done.
This means that Husky is not working at the moment.

PR #176 merged, Thanks for fixing this @pozil


Logs in kernel

Description
Fixes #589
Checklist:

[ ] I have reviewed my diff in github, line by line.
[ ] Every change is related to the PR description.
[ ] I have linked this pull request to the issue(s) that it resolves.
[ ] There are no unexpected formatting changes, superfluous debug logs, or commented-out code.
[ ] The branch has been merged or rebased against the head of its merge target.
[ ] I'm happy for the PR to be merged at the reviewer's next convenience.

Makes sense to pick up again once Adam merges https://github.com/AztecProtocol/aztec-packages/pull/634


"Find References" doesn't show the reference in .class files

Let's make sure issue is not already fixed in snapshots first.

[x] I verified I can reproduce this issue against latest snapshots build

Steps to reproduce
From a fresh installation and clean workspace:

Create a java project with a dummy java file
Make this file reference some JDK type (eg FileInputStream)

I tried

Select a location in FileInputStream token in the file
Find References

I expected: Several references in the JDK code
But got: nothing
Here is some relevant log output form (<workspace>/.metadata/.log)
The JDT communication log can show many entries with URIs of form jdt://contents/jdk.compiler/com.sun.tools.javac.api/JavacTrees.class. Those URIs don't seem to show in the result list.
Community

[x] I understand reporting an issue to this OSS project does not mandate anyone to fix it. Other contributors may consider the issue, or not, at their own convenience. The most efficient way to get it fixed is that I fix it myself and contribute it back as a good quality patch to the project.

Addressed in LSP4E: https://github.com/eclipse/lsp4e/pull/810


encoding/asn1: Marshal doesn't handle nil interface values correctly

What version of Go are you using (go version)?

$ go version go1.14.2 linux/amd64

Does this issue reproduce with the latest release?
yes
What operating system and processor architecture are you using (go env)?
go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/christoph/.cache/go-build"
GOENV="/home/christoph/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOOS="linux"
GOPATH="/home/christoph/.gopath"
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build641307088=/tmp/go-build -gno-record-gcc-switches"

What did you do?
asn1.Marshal a struct containing a nil interface with omitempty tag.
https://play.golang.org/p/x-hrKEJH0Fe
What did you expect to see?
No error and the corresponding field should be omitted from the output.
It works with nil slices (see example).
What did you see instead?
Error: asn1: cannot marshal nil value

For anyone who encounters this problem: As a workaround, you can test if the corresponding interface field is nil before marshaling and assign []byte(nil) to it.

/cc @FiloSottile @agl


avoid pre-compile time errors in expand_include_lib_path

Thanks. Do you mind extending the commit message with a previously broken include_lib directive?

No problem. Done.

This probably warrants a debug log msg.

ok

Tested the changes and the pull request works here. Thanks @tuncer.

Is this ready to merge? @ferd?

@kejv should we have tests to go with this?

Are we able to tell whether compilation failed because of rebar internal error or compile time error (I am thinking of retest_sh:run/2)?

On Tue, May 5, 2015 at 8:03 PM, kejv wrote:

Are we able to tell whether compilation failed because of rebar internal error or compile time error (I am thinking of retest_sh:run/2)?

Scanning the error output from run/2 would be the usual way.

Compare the rebar errors produced by 1) erlc, 2) rebar itself:

{stopped,{1,
          ["5396","==> erlc_dep_graph_rt (compile)",
           "src/lisp.erl:11: can't find include lib \"include/non/existent.hrl\"",
           "Compiling src/lisp.erl failed:",
           "ERROR: compile failed while processing /home/david/git/rebar/rt.work/20150506_175922/erlc_dep_graph_rt: rebar_abort"]}}

{stopped,{1,
          ["5550","==> erlc_dep_graph_rt (compile)",
           "ERROR: compile failed while processing /home/david/git/rebar/rt.work/20150506_175958/erlc_dep_graph_rt: {'EXIT',",
           "    {function_clause,",
           "        [{filename,join,[[]],[{file,\"filename.erl\"},{line,392}]},",
           "         {rebar_erlc_compiler,expand_include_lib_path,1,",
           "             [{file,\"src/rebar_erlc_compiler.erl\"},{line,696}]},",
           "         {rebar_erlc_compiler,process_attr,3,",
           "             [{file,\"src/rebar_erlc_compiler.erl\"},{line,646}]},",
           "         {rebar_erlc_compiler,parse_attrs,2,",
           "             [{file,\"src/rebar_erlc_compiler.erl\"},{line,614}]},",
           "         {rebar_erlc_compiler,modify_erlcinfo,4,",
           "             [{file,\"src/rebar_erlc_compiler.erl\"},{line,444}]},",
           "         {rebar_erlc_compiler,'-update_erlcinfo_fun/2-fun-0-',4,",
           "             [{file,\"src/rebar_erlc_compiler.erl\"},{line,399}]},",
           "         {lists,foldl,3,[{file,\"lists.erl\"},{line,1248}]},",
           "         {rebar_erlc_compiler,init_erlcinfo,2,",
           "             [{file,\"src/rebar_erlc_compiler.erl\"},{line,389}]}]}}"]}}

I don't think there's any sane way how to test the difference (and I don't consider scanning the strings in the list with regex as sane). retest_sh:run/2 should probably provide some more detailed error codes.

Doesn't that mean we should (1) print a more informative error in erlc_compiler, and (2) can't we scan for the ?DEBUG message?

(1) No, since we don't print any error message at all in erlc_compiler. Quite the contrary actually, we want to try to avoid any errors.
(2) No, since ?DEBUG message is present only if compiling on debug level and I don't think we should rely on this fact.
Moreover, I really start thinking that printing the debug message is not such a good idea. Because it will print for e.g. -include_lib("erl_compile.hrl"). but not for -include_lib("stdlib/includ/erl_compile.hrl"). (note the typo). I think we should not pretend that we are trying to catch invalid include_lib file at this place. User will get compilation error later anyway. Only thing done here is trying to prevent any rebar internal errors.

I agree, but as we're treating the included header as a dependency, shouldn't we at least log a DEBUG msg on ENOFILE?

Note that at this stage we don't know yet whether the header is a dependency for rebar logic (erlcinfo). This will be decided only in expand_file_names/2 based on content of Dirs variable. Generally I think there's no reason to duplicate the work of erlc. If the header file doesn't exist erlc will throw an error anyway.

But don't we already do the check before updating the digraph?

No. We collect all included headers (in process_attr) and then filter out those which are not in the include path (in expand_file_names) and this will typically filter out all include_lib files.

Generally I think there's no reason to duplicate the work of erlc. If the header file doesn't exist erlc will throw an error anyway.
Still this is the strongest reason why rebar should not try to check existence of files, yet even printing some related debug messages.

Okay. Let's add the previously crashing scenario as a regression test and merge this.

Until retest_sh:run/2 starts producing more detailed error codes I am afraid we will have to live without regression test...

Having fixed the rebar-internal error, why can't we (1) expect run/2 failure AND (2) scan for can't find include lib...?

You mean something like this?
{stopped,{_,Lines}} = Result,
?assertMatch([ _ | _ ], lists:filter( fun(L) ->
        re:run(L, "can't find include lib \"include/non/existent.hrl\" ) /= nomatch
end, Lines)

This seems quite hacky to me. And moreover it is also fragile. What if someone decides that he really wants that debugging message (which I didn't) and formulates it in exactly the same way as erlc? Then if he breaks the code the test will still pass. Or what if erlc for some reason changes the error message?
Frankly, I would rather have no tests at all than tests with potential of false negatives or positives.

Something like that or inttest/logging.
If someone adds that debug msg, he would adapt the test. Also, looking for a less specific error msg and having a minimal module that's unable to provoke any other error should make the scan resilient against changes in compile:file. That way, the test is unlikely to fail.
However, the missing test is not important enough to hold up the patch. So, I'm okay without it.

@ferd?

Uh yeah, so that +7-3 patch seems good then. Merging.


Fix remote() for constructors with no args

We had a short-circuit for .remote() without arguments, which isn't really needed.

Nice, this makes sense. It's possible my refactoring(s) broke this, since it seems like we didn't have any tests. Thanks for adding it!


Reduce apptimeout test delays

No sense in waiting extra time.
Type of change

Code maintenance/cleanup

related test idea for another time: check (somehow) that the timed-out process got killed. From a look at the code I don't see anything that does that... bash_app just stops waiting and reports an exception, without any further management of the launched process...


Add inference capabilities to conve, convkb, distmult, ntn, proje_pointwise, slm and sme

Following #87 , this PR has expanded inference on heads, tails and rels to conve, convkb, distmult, ntn, proje_pointwise, slm and sme models. It extracted default implementation on distance() and infer_*s() into InferenceMeta for the purpose of easy sharing and overriding. Thanks!

Also, separated out calculations on L1 norm and L2 norm into dissimilarity() used by infer_*s() and it keeps distance() clean for each model. Added infer_*s() to hole, complex, rotate and tucker. There will be extra work and care to be taken for inference with kg2e, rescal and transg so the implementation needs to be discussed and done on a separate PR.

Also, separated out calculations on L1 norm and L2 norm into dissimilarity() used by infer_*s() and it keeps distance() clean for each model. Added infer_*s() to hole, complex, rotate and tucker. There will be extra work and care to be taken for inference with kg2e, rescal and transg so the implementation needs to be discussed and done on a separate PR.

Hi, @louisccc. Is there anybody happy to have a review on this PR?

Yes I can do !

I will say its probably not a good idea to place the implementation in a metaclass (inferencemeta.py), maybe we should look for some other ways to refactor this. But its general good now.

Agreed. The InferenceMeta could be refactored into a separate concrete class (e.g., inferer.py) and that would make KGMeta.py stay true abstract albeit being up to the overall code design. Cheers!


Parallelize python-inspector during analyzer execution

Python inspector runs on a single core, parallelizing its execution could speed up large python projects.

As discussed in the community meeting, the ORT analyzer does not run any single of its analyzers in parallel currently (only analyzers for different definition files are run in parallel). In general, it's not in the scope of the ORT analyzer to implement parallelism when running an analyzer of a specific type; instead, that is the responsibility of the analyzer implementation, i.e. in this case the python-inspector. So I'm closing this in favor of the upstream python-inspector issue. When updating to a python-inspector version that has this feature implemented, we'll take care of making use of it in ORT.


Fixing the truncated error with ISO Date on MySQL

Fixing this issue: https://github.com/llambda/connect-session-knex/issues/19

Looks solid to me

Thanks for the PR! New version will be published soon.

Thanks!!


newbie question (sorry if im missing the obvious)

sorry im super new to ros and am hoping to use DWA for a time of flight sensor, so i have the same ubuntu 20.04 and ROS Noetic, but after following your instructions I get an error :
Error: The current folder is not part of a catkin workspace.
Im not sure whats wrong and am struggling to find advice on the matter. the folder is in catkin_ws/src and i have catkin_tools installed as well. i dont have any othe projects within the workspace yet tho. any advice is greatly appreciated.

hi hopefully this is some more detail to help, when i do a dry run :

If there are no other projects in the workspace, I recommend that you recreate the workspace and reinstall dwa_planner.

Recreate workspace

rm -rf ~/catkin_ws
mkdir -p ~/catkin_ws/src
cd ~/catkin_ws
catkin build

Install and build dwa_planner

cd ~/catkin_ws/src
git clone https://github.com/amslabtech/dwa_planner.git
catkin build dwa_planner

awesome thank you! I appreciate the advice greatly :)


Feature/button reg

Changes:
added button register must register buttons if you use the enable method, allows you to search for buttons by their use, can be helpful if you want to create a layout in another class
Controls and Usage:
add Devices.enableButtonRegistration() in construction before calling any other button method
use Devices.controller#.registerButton(Button, String use) to register a button, registering twice will throw an error, and failure to register will throw an error
use Devices.controller#.searchButton(String use) to search for the use registered
Reviewers:
@XaverianTeamRobotics/senior-programmers

I wont delete this branch because I want somebody to work on documenting it.


[GUI-52] Add mongoose ip as an argument to NGINX.

Add ability to pass mongoose IP as a parameter.

Base has been changed due to changing docker file in subtask of GUI-55.


Update dependency on MKFoundationKit to 1.3.x

Pod outdated says:

MKFoundationKit 1.2.3 -> (unused) (latest version 1.3.0)

It would be nice to get the dependency up to date.
Thanks for a really useful library!

Cheers. I will update it shortly.

Done.


Let flake8 ignore E203

Description
There is a conflict between black and flake8 dealing with the space before colons used for slicing. Letting flake8 ignore this will fix it.

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (9efb527) 93.83% compared to head (b05955d) 93.83%.

Additional details and impacted files
@@           Coverage Diff            @@
##           develop     #176   +/-   ##
========================================
  Coverage    93.83%   93.83%
========================================
  Files           28       28
  Lines         1395     1395
========================================
  Hits          1309     1309
  Misses          86       86

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Check for nil permissions terms during model validation

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

1 of 5 (20.0%)  changed or added relevant lines in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-0.05%) to 60.818%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

model/validate.go
1
5
20.0%

Totals

Change from base Build<PHONE_NUMBER>:
-0.05%

Covered Lines:
2794

Relevant Lines:
4594

💛  - Coveralls


abstract access control in workflow endpoints

right now we first fetch the workflow and then check if the user belongs to the project the workflow is attached to.
move this check to a separate function that checks access and returns the workflow or raises the appropriate error otherwise.

Note: this simply means a function get_workflow_check_owner, similar to the current get_project_check_owner.


⚠ Bofrost.be has degraded performance

In 96b1d40, Bofrost.be (https://www.bofrost.be) experienced degraded performance:

HTTP code: 200
Response time: 1185 ms

Resolved: Bofrost.be performance has improved in e3d33f8.


Enquire about the database splitting - 60% training, 20% validation, and 20% testing

Dear,
Thanks a lot for your great work!
Could you share your experimental results (SRCC and PLCC) (Database: LIVE, CSIQ, TID2013, and KADID-10k) by splitting the dataset into 60% training, 20% validation, and 20% testing?
Thanks in advance and have a nice day!
Best,
Shuyue
Feb 8, 2022

The experimental results are as follows:
LIVE: SRCC 0.9684 PLCC 0.9704
CSIQ: SRCC 0.9203 PLCC 0.9376
KADID-10k: SRCC 0.8846 PLCC 0.8840


CI tests are failing because netlib.org is down

Should files for the integration tests be hosted on another more reliable site or copied into the travis directory and committed to the c2go repo?

What do we need from netlib.org?

It's for the transpilation of triangle.zip (2D mesh generation, triangulation, and mesh display/printing).

Ah I see, or we can just remove it from the build process?

The inclusion of SQLite and Triangle still give interesting metric, i.e. the number of transpilation warnings, go build errors and gometalinter issues.
With that I noticed that with the development over the last months the number steadily went down. However with https://github.com/elliotchance/c2go/pull/754 it went up a bit again.
I checked and the files are 2.1 MB and 160KB respectively. So I would include them in the travis folder and commit them, thus reducing the dependence of CI on external sites (other than github).

Should I prepare a PR for this?

SQLite3 should still stay. It is the ultimate goal to work towards and I doubt downloading the source from sqlite.org will ever cause us any issues. But feel free to create a PR that deletes the triangle code.

I have created a PR where the CI dependencies are vendored in.
I think this is acceptable, since the source files are not expected to change and their size is not excessive.
E.g. for an update of the SQLite version the travis/test.sh needs to be changed anyway.
As for deleting the triangle code, I would keep at least a 2nd program in the CI  pipeline so as to have a confirmation of the trend the metrics are going (i.e. less or more transpilation warning or go build errors).

We are not recording the metics anywhere useful. The only one that matters is SQLite3 otherwise we would have lots of example packages to compile and check for warnings which just slows down the build.