split into opencivicdata.core and opencivicdata.legislative #83

as discussed in #83 - figured proving the concept would be valuable
it is a pretty clean split, there's one quirky circular dependency related to the admin, but nothing that wasn't simple to solve with some conditional behavior that checks if legislative is installed

Changelog includes the only caveats I know of:
https://github.com/opencivicdata/python-opencivicdata-django/pull/85/files#diff-4ac32a78649ca5bdd8e0ba38b7006a1eR8
basically migrating users will have to change their INSTALLED_APPS to include both apps and will have to run a fake migration

Coverage decreased (-31.5%) to 67.069% when pulling da84ab0c3fb8aaa807a2a3fcc5069f528289a400 on split-apps into b9bb9b6067764556db011cfa96bd782a869b1be6 on master.

Coverage increased (+0.2%) to 98.741% when pulling 1984418b1c6d1905ebb6c685b1862ec6aff4a69d on split-apps into b9bb9b6067764556db011cfa96bd782a869b1be6 on master.

thoughts @jpmckinney & @fgregg ?  this would be a breaking change but the steps to update aren't too bad and I do think it'll make the future better as we go beyond the legislative

Sorry, pretty covered up right now. Can you give me until middle of next week to review and think about this?

yeah, no rush on my end- just wanted y'all to see it
On May 19, 2017 17:18, "Forest Gregg"<EMAIL_ADDRESS>wrote:

Sorry, pretty covered up right now. Can you give me until middle of next
week to review and think about this?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/opencivicdata/python-opencivicdata-django/pull/85#issuecomment-302837367,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAAfYpjUkLGXeTv4CkVtcCYnlfgljmgpks5r7jFGgaJpZM4Nf-u2
.

I didn't read the diff thoroughly but it mostly looks like copy-pasting things between files, changing imports based on those moves, adding lines to ensure the database table names are unchanged, and a little try/catch in the admin to handle legislative not being present. Was there any part requiring special attention? Otherwise looks good to me.

that's all it does, I just wanted to clear the breaking change described here
https://github.com/opencivicdata/python-opencivicdata-django/pull/85/files#diff-4ac32a78649ca5bdd8e0ba38b7006a1eR8
with you as it'll affect existing installations

Yup, looks like the break is as minimal as can be :)

Shouldn't each app (core, legislative, elections, etc.) have their own set of tests?
And maybe this is related, but then wouldn't the constants and merge functions move into core?

Good point, I think there's more room to push things in that direction, but
fortunately that'll all be behind the scenes refactoring so I figured I'd
do the controversial/ public-facing stuff first.
I should find some time to move the tests into their respective modules,
I'm undecided on the merge code, since it (like divisions.py) sort of sits
outside the needs of any one django app.
On Tue, May 23, 2017 at 9:52 AM, James Gordon<EMAIL_ADDRESS>wrote:

Shouldn't each app (core, legislative, elections, etc.) have their own set
of tests?
And maybe this is related, but then wouldn't the constants and merge
functions move into core?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/opencivicdata/python-opencivicdata-django/pull/85#issuecomment-303464159,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAAfYvdEDKiHYJIkm-YPlDu46Cnaxk2Iks5r8w62gaJpZM4Nf-u2
.

Seems like all the apps will depend on core, either directly or indirectly, in which case having merge in core makes it available to all the other apps. But, yeah, incremental steps.
I'm cool with merging this as is. Then, if you like, I can help move the tests into respective apps ahead of our PR for the elections models.

👍 if this works for datamade we'll go ahead

Hi all,
So my concern is purely social and political. Up to now, having all of this in one repo hast meant that @jamesturk and @jpmckinney have been gatekeepers for the extensions to OCD that I, @gordonje, and @aepton have been pushing.
While this has slowed development down, I think that this has ultimately led to a much better models. Since we are basically, in the data standard creation business here, that seems like the really correct trade off.
I well appreciate the technical benefits of this approved split, but I can also imagine it leading to a shift in organizational dynamics. It's easy to imagine that, as a bunch of busy folks, that we will tend not to scrutinize the repos that we do not use regularly. I think this would be net loss.
Now we are really talking about governance issues, and I know that there other things that to talk about in this regard, so maybe we should have a phone conversation about governance soon. I know that @djbridges is interested in talking about some governance issues as well (particularly around ocd-ids)
So... tldr, i'd like to have a clarification of how governance of reference implementations of OCD models accompany this technical change.

Happy to join a call on governance and other socio-political questions.

Same, definitely something we should sort out.  Just sent an email to figure out timing.
As for this particular change- that concern about being able to just ignore the apps that were further from one's area(s) of interest was one of the reasons I landed on keeping this in one repo, but yeah it is def. still a risk as this grows to include more parts we aren't all using.
Are we OK with this change for now or would you rather wait?

I'm okay with it. Give me a few hours, to pin some various dependencies though.

SGTM
On Wed, May 24, 2017 at 9:04 AM, Forest Gregg<EMAIL_ADDRESS>wrote:

I'm okay with it. Give me a few hours, to pin some various dependencies
though.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/opencivicdata/python-opencivicdata-django/pull/85#issuecomment-303771142,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAAfYkgFvo7RBo_OJ7k-HahE1G0B758aks5r9FUHgaJpZM4Nf-u2
.

I'm good to go, @jamesturk


Fix (Jenkins) CI build issues involving Java 11 & Java 17

Fix current Jenkins CI build issues for SDG 2.7.x, as seen here.

This is done!


Add --rm-stale option and make robustness changes

Adding a --rm-stale option to request deletion of .tgz files in the working dir that do not correspond to images in the generated mapping.txt file
Always regenerate mapping.txt and image list files
Download image and generate tarfile in scratch dir under working dir, moving tarfile to working dir once fully written
Update image list files after image tarfile is written

/cc @browsell @alosadagrande

/cc @alosadagrande

/lgtm


Error doing job: Task exception was never retrieved (None)

The problem
HA log shows:
Logger: homeassistant
Source: custom_components/easee/controller.py:339
integration: Easee EV Charger ([documentation](https://github.com/nordicopen/easee_hass), [issues](https://github.com/nordicopen/easee_hass/issues))
First occurred: 21:06:29 (5 occurrences)
Last logged: 21:10:27

Error doing job: Task exception was never retrieved (None)

Traceback (most recent call last):
  File "/config/custom_components/easee/controller.py", line 706, in refresh_sites_state
    charger_data.check_latest_pulse()
  File "/config/custom_components/easee/controller.py", line 339, in check_latest_pulse
    elapsed = now - self.state["latestPulse"]
                    ~~~~~~~~~~^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/pyeasee/utils.py", line 55, in __getitem__
    if type(self._storage[key]) == str and validate_iso8601(self._storage[key]):
            ~~~~~~~~~~~~~^^^^^
KeyError: 'latestPulse'

Version of Easee integration having the issue?
0.9.59
Version of Home Assistant Core having the issue?
2024.9.2
Anything in the logs that might be useful for us?
No response
Additional information
No response

Same as https://github.com/nordicopen/easee_hass/issues/457

Probably solved in v0.9.60, please update and test.

I can confirm v0.9.60 solves the isssue. Thanks a lot!!!


Bridgy stuck with polling?

When I go to my Bridgy Twitter page (https://brid.gy/twitter/islayblog) it seems to be stuck at "Polling now. Check back in a minute!". It's been like that for over an hour and also says "Last webmention sent 7 hours ago" when I know there have been well over a dozen mentions Bridgy should have picked up since.
Any idea what's happening?

hey, sorry for the trouble! i'm rolling out a big change to bridgy's underlying infrastructure, and it's hitting some problems. hopefully they'll get ironed out soon. i'll keep this open until it's fixed.

Ah, sorry to hear that. Hope everything gets sorted and you don't have any further troubles. No worries, I'll just wait for all the mentions to roll in once it's all done.

Seems it's all done, at least I had a whole raft of mentions coming in a short while ago. Feel free to close if everything is fine from your side.

true! glad to hear it.


Fix compiling with CUDA_MPI=ON

The signature for MPI_Comm_size is
int MPI_Comm_size(MPI_Comm comm, int *size)

Can one of the admins verify this patch?

OK to test

OK to test

How the heck was it never caught?

Because we don't have cuda mpi on the tester so the code is not tested

Codecov Report

Merging #147 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #147   +/-   ##
=======================================
  Coverage   87.76%   87.76%
=======================================
  Files          53       53
  Lines        2199     2199
=======================================
  Hits         1930     1930
  Misses        269      269

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6c15dfd...a704c1f. Read the comment docs.


Cannot convert Hugging face Japanese GPT to FasterTransformer model

Branch/Tag/Commit
main
https://github.com/NVIDIA/FasterTransformer/tree/35989aa49e47f9ce820032ff08807b8a228f0b68
Docker Image Version
tritonserver_with_ft:22.12 (local build)
GPU name
T4
CUDA Driver
525.78.01
Reproduced Steps

Build fastertransformer_backend

# ref. https://github.com/triton-inference-server/fastertransformer_backend/tree/283286958993a5a03d80be956521770cad1fa2be
git clone https://github.com/triton-inference-server/fastertransformer_backend.git
cd fastertransformer_backend
export WORKSPACE=$(pwd)
export CONTAINER_VERSION=22.12
export TRITON_DOCKER_IMAGE=triton_with_ft:${CONTAINER_VERSION}

python3 docker/create_dockerfile_and_build.py --triton-version 22.12

git clone https://github.com/NVIDIA/FasterTransformer.git

docker run -it --rm --gpus=all --shm-size=4G -v $(pwd):/ft_workspace -p 8888:8888 tritonserver_with_ft:latest bash

cd /ft_workspace/FasterTransformer/examples/pytorch/gpt/utils/

python3 huggingface_jp_gpt_convert.py -o /ft_workspace/models/rinna-gpt -i rinna/japanese-gpt-1b -i_g 1

You will see this error.

Traceback (most recent call last):
  File "huggingface_jp_gpt_convert.py", line 194, in <module>
    split_and_convert(args)
  File "huggingface_jp_gpt_convert.py", line 105, in split_and_convert
    config["gpt"]["model_name"] = "gpt" if hf_config["_name_or_path"] == '' else hf_config["_name_or_path"]
NameError: name 'config' is not defined

Simply define config to fix it.
That was the first error. huggingface_jp_gpt_convert.py has three issues. Please see the comment.

After resolving the config-not-define bug, you will encounter an error for incorrect variable name. The fix is easy. You just rename output_dir to saved_dir.
Traceback (most recent call last):
  File "huggingface_jp_gpt_convert.py", line 195, in <module>
    split_and_convert(args)
  File "huggingface_jp_gpt_convert.py", line 117, in split_and_convert
    with open(output_dir + "/config.ini", 'w') as configfile:
NameError: name 'output_dir' is not defined

Finally, after fixing the variable name, the code will run and exit without problem, but it will produce incorrect names for model weights as well as some weights gone missing. The resulting model names are in model.h.* pattern, while the correct one should be model.layers.*. Here's the step to reproduce.
# continue from steps reported above, you're still inside a container of tritonserver_with_ft:latest
1. `mkdir /ft_workspace/models/configs; cp -r /ft_workspace/fastertransformer_backend/all_models/gpt/* /ft_workspace/models/configs
2. `vim /ft_workspace/models/configs/fastertransformer/config.pbtxt`. Change the `string_value` of `model_checkpoint_path` in line 288 to `"/ft_workspace/models/rinna-gpt/1-gpu"`
3. CUDA_VISIBLE_DEVICES=0 /opt/tritonserver/bin/tritonserver  --model-repository=/ft_workspace/models/configs/

You will see the following errors because of the incorrect model name pattern.
[FT][WARNING] file /ft_workspace/models/rinna-gpt/1-gpu//model.layers.23.mlp.dense_h_to_4h.bias.0.bin cannot be opened, loading model fails!

[FT][WARNING] file /ft_workspace/models/rinna-gpt/1-gpu//model.layers.23.mlp.dense_4h_to_h.bias.bin cannot be opened, loading model fails!

[FT][WARNING] file /ft_workspace/models/rinna-gpt/1-gpu//model.layers.23.attention.query_key_value.weight.0.bin cannot be opened, loading model fails!

[FT][WARNING] file /ft_workspace/models/rinna-gpt/1-gpu//model.layers.23.attention.dense.weight.0.bin cannot be opened, loading model fails!

[FT][WARNING] file /ft_workspace/models/rinna-gpt/1-gpu//model.layers.23.mlp.dense_h_to_4h.weight.0.bin cannot be opened, loading model fails!

(and many more)

There are two things to fix.

Line 176; Use state_dict().items() instead of named_parameters().
Even tho you fix 1, some weights are still missing.

[FT][WARNING] file /ft_workspace/models/rinna-gpt/1-gpu//model.wpe.bin cannot be opened, loading model fails!

[FT][WARNING] file /ft_workspace/models/rinna-gpt/1-gpu//model.wte.bin cannot be opened, loading model fails!

[FT][WARNING] file /ft_workspace/models/rinna-gpt/1-gpu//model.final_layernorm.bias.bin cannot be opened, loading model fails!

[FT][WARNING] file /ft_workspace/models/rinna-gpt/1-gpu//model.final_layernorm.weight.bin cannot be opened, loading model fails!

[FT][WARNING] file /ft_workspace/models/rinna-gpt/1-gpu//model.wte.bin cannot be opened, loading model fails!

To fix this. Line 24; from transformers import GPT2LMHeadModel instead of GPT2Model. Also change line 120 to use GPT2LMHeadModel instead. GPT2Model doesn't load those weights needed for FasterTransformer model.

It is welcome to send a fix PR, thank you.

Thanks! Btw, I'm not sure why you need FasterTransformer/examples/pytorch/gpt/utils/huggingface_jp_gpt_convert.py
when you have huggingface_gpt_convert.py. If it's for converting Japanese models from https://huggingface.co/rinna,  huggingface_gpt_convert.py can convert them just fine.

Thanks! Btw, I'm not sure why you need FasterTransformer/examples/pytorch/gpt/utils/huggingface_jp_gpt_convert.py when you have huggingface_gpt_convert.py. If it's for converting Japanese models from https://huggingface.co/rinna, huggingface_gpt_convert.py works just fine.

I remember there are some differences between them. But if you can convert the checkpoint by huggingface_gpt_convert.py, you can also update it in document.


riot-redis live replication does not sync keys after boot

Context
I am trying to replicate a live redis (redis) instance into second one that uses ssl (rediss).
Whenever I start live replication with the command:
ubuntu@ec2:~/riot-redis-2.2.7/bin$ ./riot-redis --info -h redis_host -p 6379 --db=0 replicate -h rediss_host -p 6379 --db=0 -t --live
Opening writer
Opening reader
Replicating from redis://request1-ci.gv5c2n.ng.0001.euw1.cache.amazonaws.com?clientName=riot  ? % │█│ 54

If I start rediss with an empty key set, then after riot-redis has started, most keys have been replicated from redis to rediss.
Running the same command with --debug gives me a constant stream of:
13:46:35.935 FINE io.lettuce.core.protocol.DefaultEndpoint	: [channel=0xdefcef92, /<IP_ADDRESS>:52564 -> rediss_host/<IP_ADDRESS>:6379, epid=0x4] flushCommands() Flushing 0 commands

Problem
After running the command and letting it run as a daemon, any subsequent write to redis is not replicated to rediss as expected by the use of --live.
Additional version info

riot-redis: v2.2.7
redis: 3.2.10
redis with ssl: 3.2.6

Do you have keyspace notifications enabled on the source instance? If so, do you see notifications being sent to the pubsub channel on that instance?

That was the case, thanks for your quick and specific answer. It solved my issue.


"crictl pull " of a cross-built image on a non-x86 platform fails to unpack

Description
 Consider an image that has been cross built for a non-x86 platform on a x86 platform. Eg:
You are on x86 and you are cross building images for arm64, ppc64le etc. Then when you "crictl
pull .." such images on say ppc64le platform, it fails to unpack because of the comparison of the arch specified in the manifest is done with the host arch, which will fail due to mismatch.
Steps to reproduce the issue:

ppc64le-master:: # crictl pull  gcr.io/kubernetes-e2e-test-images/nonewprivs-ppc64le:1.0

Describe the results you received:
FATA[0000] pulling image failed: rpc error: code = Unknown desc = failed to get image information: failed to get image diffIDs: manifest sha256:3639c7b61bc1118bdbd4b7a2713e4b75a7a92be578d1497f640b8abf9ef02756: not found
Describe the results you expected:
INFO[0006] PullImage "gcr.io/kubernetes-e2e-test-images/nonewprivs-ppc64le:1.0" with auth config nil
INFO[0006] resolving
INFO[0006] resolving
INFO[0008] PullImage "gcr.io/kubernetes-e2e-test-images/nonewprivs-ppc64le:1.0" returns image reference "sha256:9c856c6118717c706969ca4f45038622769ddf19d07e886f3fddb4e7fc0293e4"
Image is update to date for sha256:9c856c6118717c706969ca4f45038622769ddf19d07e886f3fddb4e7fc0293e4
Output of containerd --version:
containerd github.com/containerd/containerd v1.1.0-rc.2.m f630d5f0a639d7d73a806f19f1a6157e768756a5.m

manifest-tool inspect gcr.io/kubernetes-e2e-test-images/nonewprivs-ppc64le:1.0
gcr.io/kubernetes-e2e-test-images/nonewprivs-ppc64le:1.0: manifest type: application/vnd.docker.distribution.manifest.v2+json
Digest: sha256:3639c7b61bc1118bdbd4b7a2713e4b75a7a92be578d1497f640b8abf9ef02756
Architecture: amd64
OS: linux
# Layers: 5
layer 1: digest = sha256:b0ead73150358c8bb3aa3a7afb6244a8f6167dbda0bdd8cd2914ac852af3e4f8
layer 2: digest = sha256:f5286b88dc4825e1af36e9566577c24b9fbcab4c33d74d25e89de3a1520c3c29
layer 3: digest = sha256:45f6fd2ea6a08103e385ddb12991ca61442b1f396f14bdb11efec764a6e7aac5
layer 4: digest = sha256:802f829d6ea9fdf49c08e999f96755902000ddde865a88154de96a49cf9eb491
layer 5: digest = sha256:75845febaa5e16e7f927b70a2974fd61922f29cf4f2118e247cca94a5b0ad24c

Yes currently cross built images are labelled with the host architecture, and at least docker build currently has no way to override, and I don't know of other builders that do yet. If Docker pulls them via a manifest list it works ok but not if you pull directly, but that behaviour is somewhat accidental and I would not expect other runtimes to do that (although it is useful in the short term).

@justincormack @crosbymichael
So you mean this issue will not be fixed in crictl, right? For image build across arch with docker build, you still recommended to use docker pull instead?

We are also facing this issue. crictl pull errors out when a ppc64le image, built on x86 arch, is pulled on ppc64le platform.

The docker buildx plugin can correctly label cross built images, using buildkit, and this functionality will be in docker build in future.


[Bug]: Train Conformer ASR for Librispeech get bad results without CTC loss

Describe the bug
Hi,
In the Librispeech recipe, I have tried training the Conformer ASR without CTC loss. So I set the ctc weight to 0 in the parameter file:
ctc_weight: 0.0
grad_accumulation_factor: 2
max_grad_norm: 5.0
loss_reduction: "batchmean"
sorting: descending

For the decoding parameters I have the following settings:
min_decode_ratio: 0.0
max_decode_ratio: 1.0
valid_search_interval: 10
valid_beam_size: 10
test_beam_size: 10
lm_weight: 0.60
ctc_weight_decode: 0

And I use !new:speechbrain.decoders.S2STransformerBeamSearch for validation and test search.
I trained 30 epochs on the Librispeech clean 100 data.
By observing the training curve, I found that the train loss was decreasing all the time during training, but the validation loss was not decreasing.

The final test result is also very bad : Epoch loaded: 30 - test loss: 1.09e+02, test ACC: 1.52e-01, test WER: 99.13
I would like to ask if there are some settings in my decoding parameter that need to be changed to match the ctc weight of 0?
Expected behaviour
I would like to know why there is such a big difference in the loss of train and validation, hoping to achieve more normal training results for Conformer ASR without using ctc loss.
To Reproduce
id=0,1,2,3
CUDA_VISIBLE_DEVICES=$id python train.py hparams/conformer_small.yaml --data_parallel_backend

Versions
No response
Relevant log output
No response
Additional context
No response

Hi can you try with regularMHA for positional encoding and report the results here please?

Hi can you try with regularMHA for positional encoding and report the results here please?

Hi, I got similar results with this parameter:

Epoch loaded: 30 - test loss: 2.21e+02, test ACC: 1.52e-01, test WER: 96.54

The problem comes from the batch size, 12 is not big enough for attention to learn the alignement in that setup. You should try having an effective batch size around 100 either by increasing the grad accum or by increasing the BS. I would encourage you to use DDP instead of DP as well as encouraged by PyTorch.

The problem comes from the batch size, 12 is not big enough for attention to learn the alignement in that setup. You should try having an effective batch size around 100 either by increasing the grad accum or by increasing the BS. I would encourage you to use DDP instead of DP as well as encouraged by PyTorch.

Not easy to get to that value, but thanks for the reply!

grad_accum won't affect your VRAM consumption. So if you have a smallish GPU, just play with that. If it's with dynamic batching, indeed, things become a bit messy ... just here, just try to make sure that "most" batches are big enough: above 64.

Hi,
any news regarding this issue please? :)

Hi,
any news regarding this issue please? :)

Hi,
I reused a larger train set (Librispeech clean 100 + 360) and a larger grad_accum value (with batch size 4, 4 GPUs, gradient accumulation value 10) for no ctc training. I did get a better result and I found that in this case it was necessary to increase the number of heads of attention appropriately, I increased the number of heads from 4 to 8. Here are the experimental results：

the test result is 14.65% for test clean set. Of course there is still a gap between this result and the expected result, and I am trying to find a solution from adjusting the training scheme (for example, changing the LR according to different epoch stages, because I found that the default LR schedule's don't change the LR very much)

Hi, do not forget to pull the latest develop. I fixed a bug with conformer.

Hi, do not forget to pull the latest develop. I fixed a bug with conformer.

Thanks for your reminder! I reran the same experiment on the latest version and after 100 epochs, I obtained a WER of 5.38%. The test results using CTC loss were 4.95%. I think I have achieved satisfactory results and I am very grateful for your patient help!

I am assuming this issue is resolved so I will close it as such, feel free to reopen if I'm wrong.


Add reverse toggle action

As requested by a review on the JetBrains marketplace.

It would be beneficial to include shortcuts for shuffling items up and down. When dealing with numerous suggestions, navigating through them becomes more convenient by moving both upward and downward.

This could be implemented as a simple reverse toggle action.
The current shortcut for the toggle action is Ctrl/Cmd+Shift+X. Using an additional modifier (e.g., Alt/Opt) for the reverse action might make it cumbersome to activate. However, I'll likely go with that for now anyway (so it will be Ctrl/Cmd+Shift+Alt/Opt+X). Better keybind suggestions (which do not conflict with the default IDE's keybinds) are very much welcome!

Implemented as per:

#58

Thank you for undertaking this task. It greatly streamlines the use of the dictionary, and I am extensively utilizing it. I'm really pleased that you took the initiative.


FMS-2 Storage / retrieve files based on tenants ID not on environment

Currently we use environment for storage / retrieve file, which is not so OK.

implemented 1.0.1


fix(ollama): TypeError

Ollama | TypeError: 'async for' requires an object with aiter method, got bytes #1024
llm:
  api_type: ollama
  model: qwen:72b-chat
  base_url: http://<IP_ADDRESS>:11434
  api_key: x

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 82.73%. Comparing base (eb8e846) to head (7b6c5ef).
Report is 108 commits behind head on main.

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@           Coverage Diff           @@
##             main    #1025   +/-   ##
=======================================
  Coverage   82.73%   82.73%
=======================================
  Files         224      224
  Lines       13135    13135
=======================================
  Hits        10867    10867
  Misses       2268     2268

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

we have given the example of base_url: http://xxxx:11434/api in the document.


chore: fmt

run cargo fmt

#499  👈
master

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @pepyakin and the rest of your teammates on  Graphite


JsonWebKeyConverter - ConvertFromECDsaSecurityKey for netstandard 2.1

I was missing an option to convert ECDsa SecureKey to JsonWebKey at JsonWebKeyConverter.cs.
But I figure out that the method actually exists, ConvertFromECDsaSecurityKey, but is only available to netstandard 2.0.
There is any plans to move on an make it also available for netstandard 2.1 ?

@brunohbrito if you look at our dev branch (6.x) we have been adding some such needed features (along with SignedHttpRequest) that we are planning on releasing soon.
In addition, we are preparing a clean netstandard 2.0 crypto layer.
There are some breaking changes that we are currently sorting out.

@brunohbrito i am going to close this and reference this issue in: https://github.com/AzureAD/azure-activedirectory-identitymodel-extensions-for-dotnet/issues/1290


Accept relative path as env variable - CMake

When I tried to set up CMake and defined IDF_PATH relative to user's home ~/esp/esp-idf' I got an error:  No such file or directory: '~/esp/esp-idf/requirements.txt' in check_python_dependencies.py as with open("~/esp/esp-idf) requires full file path.
Fix expands IDF_PATH with user home if it starts with ~

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

Hi @domenbukovac,
Thanks for sending this contribution.
I have some concern this may fix the immediate problem with having a ~ in IDF_PATH but move it further down into other parts of the system, as paths containing IDF_PATH may get passed into other Python scripts by CMake.
Did you find that everything else works OK after applying this change?

@domenbukovac Thanks for the contribution. Would you please help share if any updates regarding @projectgus question "Did you find that everything else works OK after applying this change?" ? Thanks.

I didn't experience any problems so far on Mac OS.

Adding my two cents to this. Expansion of ~ is really shell thing and as such I don't see it as a good thing being used in IDF_PATH. Having just done a lot of work re. CMake in my own project, I have no hard time imagining it becoming an issue.
I'm curious just how you set your environment variable, @domenbukovac as this yields the correct result:
export IDF_PATH=~/esp/esp-idf
echo $IDF_PATH
/home/permal/esp/esp-idf

Did you set IDF_PATH in your IDE? If that's the case then the underlying issue is that the IDE doesn't expand it when passing on the environment to the build.

Closing as @PerMalmberg's reasoning seems correct in this case, and there hasn't been any other update. (For the record: I also set IDF_PATH with ~ but I do it in a way that it's expanded by the shell and not pushed into CMake or idf.py).


Forego pages, only use pagelets

During my last presentation @AmsterdamJS this was suggested by a listener. Based on the frameworks we showed of pages entirely consisting out of pagelets. The driving force behind this change is the ability to have recursive pagelets. If pagelets can act as container, why would pages be required?
This issue could have been created on the pagelet repo as the biggest changes will be made to that code/repo, however this is more about the general pattern behind bigpipe.
Currently pages are responsible for the following. The question/discussion is where should each part of the logic go? Either shift it to bigpipe server or the pagelet.
Pagelet discovery
The pages that are provided to the Pipe instance will automatically discover their pagelets on initialization. Pagelets already contain the traverse function to discover recursive pagelets, in theory this could be swapped 1:1.
Rendering
Pages have logic to accommodate each different rendering mode, e.g. async/pipeline/sync. https://github.com/bigpipe/bigpipe/blob/master/page.js#L297-L396. This logic can be shifted to pagelets.
Configure/bootstrap html response
Renders the initial headers and writes the HTML to the client as fast as possible. This is external to pagelets and should move to the bigpipe server. To accommodate request dispatching, the request handling should ideally also be done by the bigpipe server.
Request handling
Pipe.dispatch is responsible finding the Page with the correct route and invoking the Page.configure. This logic could be shifted to the Pagelet.configure method, as long as Pagelets have routing. The write, flush and end can be shifted to the bigpipe server. Each pagelet has a reference to the pipe instance and should be able to call these methods with the correct data (in stead of referencing this).
Routing
Any request is dispatched from bigpipe to responding routes. In stead of the page having a router the pagelet can have a Router instance. The tricky part is to ensure this is only done for parent pagelets. When the bigpipe instance calls optimize on parent pagelets this should be handled. The parent pagelet will also have to maintain a params property.
View and dependencies
Pages and pagelets already both have a view template, this should not matter. In fact by not using a templater (but client side JS, if async render) the amount of template code in templates should be reduced. Since pagelets can simply be included using <element data-pagelet="name"></element>
Charset, status code and content-type
Should just as easily be contained by the pagelet, the parent pagelet will then determine these, even if a child pagelet has different values. Those child pagelet values shoud be ignored.

Most bases are covered by the current refactor

Closing based on comment above as we shipped 0.9 which includes this refactor.


Screencast video is not up to date

Can we update the screencase video to match current screencast.sh and the updated clearlinux-demo-opae demo?
The current video shows using intel-fpga-demo-compress:devel which doesn't exist in the repo.

@mythi You may close this issue as the screencast video has been updated


Add a SnowplowSubject class equivalent to remove need to get same data over and over

Currently to get extra information for decorating events we are querying for it on every single event.  This is a huge waste of resources that can be removed by simply getting all of the information once and storing it in a pre built dictionary on creation.

Great idea!


[v3] Can't set background-color in sass

My background colour is coming from reboot.css:
https://github.com/coreui/coreui/blob/v3-next/css/reboot.css#L26
Rather than what I set for $body-bg, which looks like I should be able to customise it per the docs here: https://coreui.io/v1/docs/getting-started/options/
I can't find any equivalent documentation page for V3.

Fixed in v3.0.0-rc.3


spk should allow for replacement of the traefik deployment with something custom

As a: SRE, I might need to replace the traffic HLD deployed by SPK with a custom drop in.
I want: Currently, if I want to replace the traefik2 deployment with something custom, I must manually edit the files in the repository for the HLD pipeline.
SPK should allow for specifying a custom component.

This is already supported via spk hld init
Usage: hld init|i [options]
Initialize High Level Definition repository. Add manifest-generation.yaml file to working directory/repository if it does not already exist.
Options:
  --default-component-git <component-repository-url>  The default hld repository's component's git repository url. (default: "https://github.com/microsoft/fabrikate-definitions.git")
  --default-component-name <component-name>           The default hld repository's component's name. (default: "traefik2")
  --default-component-path <component-path>           The default hld repository's component's path. (default: "definitions/traefik2")```


Yatin: Watch customroms.in

Yatin requests the watch of the watch_keyword customroms\.in. See the MS search here and the Stack Exchange search in text, in URLs, and in code.

customroms\.in has been seen in 0 true positives, 0 false positives, and 0 NAAs.

Approved by Spevacus in Charcoal HQ

From a spammer's profile.


TypeScript 3.9 lib.d.ts updates

[ ] Ensure that PRs are merged in.
[ ] Sync with https://github.com/Microsoft/TSJS-lib-generator/

It’d probably be a good idea to do https://github.com/microsoft/TSJS-lib-generator/issues/826.

This is done now.


Test framework -  upload test

@david-yz-liu @WilsonChiang
Builds on top of #2321 which is to be merged in and fixes the bug with polymorphic urls.
This PR allows us to upload a test file for a given assignment.
Cases around test file uploading still to be considered:

replacing a test file
uploading multiple test file (or changing the form such that it only allows uploading of one test file)
modifying the form to render already uploaded files properly

@jeremygoh @WilsonChiang One of you should fix the Hound comments here.

@jeremygoh I'm fixing the hound comments currently.

@jeremygoh Hmm.. should I be able to push to your branch? Seems like I don't have permission.

@WilsonChiang You can't push to each other's forks unless you set the permissions for it. You can always make a pull request into Jeremy's branch.

@david-yz-liu a lot of the code that Hound has commented on, and also the block you mentioned, are pieces of code from the original test-framework branch. Should I make the suggested modifications in this PR or in another?


How to maintain multiple primary keys in a model in redis_om?

Is there any other way to store  multiple primary keys in a model ?

Why do you need multiple primary keys?


Simplify NC PR

The NC PR has been open for months, maybe it helps if we simplify it, that's to say, don't improve the code structure, but just add our share type to each switch and each if-else chain.

When done, ping NC customers within the consortium so they can push for it to be reviewed.


Publish

Learn Build status updates of commit a2e4aff:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

rd/rockford-lhotka.yml
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the learn.microsoft.com contributor guidesPost your question in the Learn support channel


on startup: error reading file "scss_tests.scss"

=> Errors prevented startup:

While building package 'meteor-scss':
scss_test.scss: Scss compiler error: error reading file "scss_test.scss"

=> Your application has errors. Waiting for file change.

I had to comment out line 17 of this package.js to get it work
// api.add_files(['scss_tests.scss', 'scss_tests.js'], 'client');

The permissions seems to be right. maybe you have an idea what's going wrong.

This isn't relevant anymore, since the build plugin has been completely rewritten.


Add extension to open-vsx.org

Hey @svsool,
thanks a lot for your extension, really enjoy using it (over dendron & foam). I'm using VSCodium which is connected to Open VSX Registry.
Can you publish your extensions to Open VSX Registry too? It's simple and straightforward according to How to Publish an Extension.
Thanks! 👍

Hi @CrCs2O4,
I'm not a user of VSX Registry.
Would you be up to contribute by modifying ci.yml to support it?
I'll provide you with the token via GH secrets.

I would prefer not to open PR without actually testing it 🙂
There is recommended GitHub Action for Publish VS Code Extension can you check it out? I assume you can directly copy-paste to ci.yml from it.
Thanks! 👍

That action seems like a super thin wrapper around ovsx, so maybe not that useful.

Awesome, thanks a lot! 👍  can you also publish your other markdown-kit extension?


Time-series widget not saving filter state in embed maps

Context
When a use changes the filters in embed maps, the state is saved in the url. In the histogram widget, if you change the filter, copy the url and open it in another window, it keeps the filter, with the time-series widget, in the other hand, it's not keeping it.
Steps to Reproduce

Create a time-series widget and publish it
Visit the embed map and filter the widget
Copy the URL and open it in another tab/window

Current Result
The filter is not applied
Expected result
The filter is applied
Browser and version
Chrome Version 59.0.3071.115
Additional info
It seems only happens with time-series widget

This is already fixed!


Connect to gfx mesh

This PR adds the wiring to visualize the meshes in pygfx, making changes appear automaticaly. I implemented this with the concept of a tracker that can be registered and that will be notified with changes. This seemed like a good approach because it allows keeping things modular (as opposed to e.g. subclassing). But we can discuss :)
Another thing that was added was that the DynamicMesh calculates the normals and also keeps them up to date.

I'm just going to merge this now. We can still discuss things and adjust later.


UnifiedNlp does not have Location to test Geocoder / no last location: Android 8.1

Hi all,
I have successfully registered UnifiedNlp in the system (that included copying Gmscore.apk into /system/priv-app

But on the microg self check it says (please see screenshot)

Unified has no known location
Unified does not have a location to test Geocoder.

I have GSM Location Service & Mozilla Location service enabled as backends. They both have location permissions. I also have Nominatim installed as the address lookup back end.
GPS is working on the device (I can use OsmAnd+ on the device successfully).
I am running CarbonROM on Android 8.1

I get the same error when using the OSM backend for Nominatim. The MapQuest backend works fine.

What happens if you got to MLS options, and/or deactivate/reactivate that backend ? Do you notice any change ?
From my experience, the MLS backend (and other maybe) sometimes seems to crash silently, prompting, the result you hsow in MicroG self-check. Going to MLS options apparently starts it again and may solve the issue.

Solution? ----> https://github.com/microg/android_packages_apps_GmsCore/issues/366#issuecomment-427252345

Not for me, unfortunately, on my device running LOS for microG (Android 8.1). Trying the pm grant  is giving me the Operation not allowed (as described in two other comments), with no change in functionality. Also tried switching backends (MLS, GSM), didn't help. Need to check logcat when I'm back home, hoping for a clue. Self-Check screen looks exactly like in the above screenshot.
No such issue on my other device, also running LOS for microG but Android 7.

@IzzySoft, you should be taking this up with the LOS microG DEV....
As long as a phone is rooted, the commands should work, that simple....

@0pLuS0 first, I'm not the only one where they throw that error – and second, the app already shows having those permissions granted. So this IMHO cannot be the cause.
Besides, when switching to network location only ("energy saving" location mode), the check gives an "all green" (suggesting network location works). But still, it gives no network location. Just updated to the latest microG version, situation unchanged.


UIWebView is deprecated

i'd want to submit my app in app store but i have an error that says:
"apple stop accepting submission of apps that uses UIWebView". the only part of my code that may use UIWebView is your sdk. please upgrade code with wkwebview

Thanks for the report,
this will be addressed in the next version.

Any word on when that new version will be? This will block us from submitting to apple

fix ---
`#import "UCWebViewController.h"
#import <WebKit/WebKit.h>
@interface UCWebViewController ()
@property (nonatomic, copy) void (^cancelBlock)();
@property (nonatomic, strong) WKWebView *webView;
@property (nonatomic, strong) NSURL *url;
@end
@implementation UCWebViewController

(id)initWithURL:(NSURL *)url cancelBlock:(void(^)())cancelBlock {
self = [super init];
if (self) {
_url = url;
_cancelBlock = cancelBlock;
}
return self;
}

(void)viewDidLoad {
[super viewDidLoad];
self.webView = [[WKWebView alloc] init];
[self.view addSubview:self.webView];
NSDictionary *views = @{@"webView":self.webView};
[self.webView setTranslatesAutoresizingMaskIntoConstraints:NO];
NSArray *horizontal = [NSLayoutConstraint constraintsWithVisualFormat:@"H:|[webView]|" options:0 metrics:nil views:views];
NSArray *vertical = [NSLayoutConstraint constraintsWithVisualFormat:@"V:|[webView]|" options:0 metrics:nil views:views];
[self.view addConstraints:horizontal];
[self.view addConstraints:vertical];
[self.webView loadRequest:[NSURLRequest requestWithURL:self.url]];
UIBarButtonItem *cancelButton = [[UIBarButtonItem alloc] initWithBarButtonSystemItem:UIBarButtonSystemItemCancel target:self action:@selector(close)];
self.navigationItem.rightBarButtonItem = cancelButton;
}

(void)close {
if (self.cancelBlock) self.cancelBlock();
[self dismissViewControllerAnimated:YES completion:nil];
}

@end`

i forked the fix here for anyone who wants it
https://github.com/felixdkatt/uploadcare-ios

Hey @felixdkatt, @yaamoo. We've just released new uploadcare-swift integration, 0.1.0-alfa. I hope you can check it out. Feedback is very welcome. If you have any feature requests or questions, shoot!


question about snapshots in open-local

Question

I am wondering how to use snapshot in open-local? Is there any instruction I can follow up ?

It seems these two restrictions make the snapshot not very useful.
Is any plan to do this part in the future?

We use read-only snapshot to backup data. We planned to support readwrite snapshot before, but have no time for it. We need more contributor to make open-local more powerful.


bastions, chimeres

This pull request was generated by the 'mq' tool
[test]
flake rate: 0.4
logical conflict every: 100
sleep for: 300s
[pullrequest]
requests per hour: 30
deps=[b,c]

/trunk merge


SameDiff: improve training and listeners

The SameDiff training api could use a few improvements (mostly validation data), and the current listener functionality (set for the SameDiff instance, no feedback, training only) isn't ideal.
Proposals:

fit methods that take in validation data and frequency, helper methods to split a DataSetIterator etc into training and validation data (e.g. give % validation), maybe KFold as well.  Make the results available to listeners.  Also add fit methods that take the validation split and use the helpers.
More provided listeners, use one for logging (ScoreListener?) and have it added by default.
Be able to pass additional listeners to fit/eval/output that are only used for that call (like how keras does it).
Update listener api to take in training stage (TRAINING/EVALUATION/INFERENCE), and to define which stages they are active for (all by default).  Also use validation results (if provided).  Listeners could also define variables they care about, which would need to be calculated each execution (e.g. if you want to report on some special value that wouldn't normally be calculated by the graph).
Make epochEnd return a boolean, if a listener returns false, stop training (for things like early stopping based on validation data).  Could do context.stopTraining() on a provided context, which could avoid accidents, but as that is the only thing the context would be used for using the return value is easier.
Add a History listener that is present by default and records loss values, fit returns this (like Keras).  Additional values could be extracted by adding additional History listeners and using the recorded values.

Maybe also add DataSetIterators to adjust batch size?
Example:

...
SDVariable accuracy = ...;
...

DataSetIterator data = ...;

// 75/25 training/validation split
DataSetIterator training = TrainingUtils.getTrainingData(data, 25);
DataSetIterator validation = TrainingUtils.getValidationData(data, 25);

// record accuracy every 5 epochs
History accuracyHist = new History(accuracy, 5);

// stop if accuracy gets over 95% on validation data and then decreases for 3 epochs, restoring best weights
EarlyStopping es = new EarlyStopping(accuracy, true, 0.95, 3, true);

History lossHist = SD.fit(training, 10, validation, accuracyHist, es);

graph(lossHist.recordedHistory());
graph(accuracyHist.recordedHistory());

@AlexDBlack if this looks good, I'll go ahead and implement it.

fit methods that take in validation data and frequency

Sounds good

helper methods to split a DataSetIterator etc into training and validation data (e.g. give % validation), maybe KFold as well.

No to both. Splitting and KFold is something I want in the data pipelines only, not something the net should ever have to worry about, for now at least.
We have DataSetIteratorSplitter already, but it's an ugly hack, and it can't be used for a lot of underlying iterators (anything with random order, basically... and it'll only fail at the start of the 2nd epoch when it detects it's invalid). We'll be overhaling splitting, k-fold etc in future planned DataVec work (DataVec 2.0). When we can do it properly, I'd be fine with adding a split arg.

More provided listeners, use one for logging (ScoreListener?) and have it added by default.

Specifics? We have score and checkpoint listeners. What else?
Adding by default - no, it's opaque behaviour IMO - the "normal logging" behaviour will disappear when users set any other listeners (i.e., they won't realize it's from a listener).
And there's no way to know an appropriate frequency for different nets (1ms per iteration vs. 10 seconds per iteration - huge spam vs. never reported, for same frequency). And disabling it is opaque (have to do setListeners(null) or something)

Be able to pass additional listeners to fit/eval/output that are only used for that call (like how keras does it).

hm... for evaluation and output, I can see that being useful.

Update listener api to take in training stage (TRAINING/EVALUATION/INFERENCE), and to define which stages they are active for (all by default).

Sounds good. I'd put it in the "At" object rather than a new field.
https://github.com/eclipse/deeplearning4j/blob/master/nd4j/nd4j-backends/nd4j-api-parent/nd4j-api/src/main/java/org/nd4j/autodiff/listeners/At.java

Also use validation results (if provided).

Not following

Listeners could also define variables they care about, which would need to be calculated each execution (e.g. if you want to report on some special value that wouldn't normally be calculated by the graph).

Hm... portability is an issue here for listeners.
There are a few valid use cases for this, so sure, sounds reasonable.

Make epochEnd return a boolean, if a listener returns false, stop training (for things like early stopping based on validation data). Could do context.stopTraining() on a provided context, which could avoid accidents, but as that is the only thing the context would be used for using the return value is easier.

I like this. But I'm tempted to make this an enumeration:
(a) Less risk of misinterpreting what it actually does (not everyone reads the javadoc, unfortunately)
(b) We can add other behaviour (though I can't think of anything other than continue/stop... and I'd rather keep checkpoints in the dedicated listener)

Add a History listener that is present by default and records loss values, fit returns this (like Keras). Additional values could be extracted by adding additional History listeners and using the recorded values.

Returning a history object from fit ops - sounds good. :+1:

Maybe also add DataSetIterators to adjust batch size?

No, not yet. We'll do it properly in DataVec 2.0
As for doing evaluation in the same training loop, and reporting evaluation back to the history object: hm... fit overloads for that sound OK.
I think I'd prefer a slightly different design though, something along the lines of:
History hist = SD.fit(training, 10, validation, es, new Evaluation());

SomeListObject acc = hist.getEvaluation(Evaluation.Metric.ACCURACY)
Evaluation e = hist.getEvaluation(

SomeListObject would basically be a List<Pair<At,Double>>, but without all the generics.
Internally in the history, we'd keep the full Evaluation objects - so user can get whatever metrics they want (and stuff like ROC curves and confusion matrices - not just doubles etc)
We'll need to think about the API here a bit more though. Consider a multi-task net: you might want to do Evaluation on one output, and regression on another...
One option is do something like this:
History h = sameDiff.fit(training)
	.numEpochs(10)
	.testData(testSet)
	.listeners(...)
	.evaluate("softmax", labelNum, 0, new Evaluation(), new ROC())
	.fit()

Basically a builder for what we'd use keyword args for in python...
I also don't want an "Accuracy" SDVariable in the graph. One design difference between us and TF is that we don't add lots of random nodes to the graph that has to be stripped out later for inference/deployment - I much prefer to keep the graph structure clean all the time, and do stuff like this imperatively/externally. IMO it's easier to develop and debug, and results is less opaque behaviour when you're doing things like transfer learning or using someone else's graph (or even your own when you forgot you added training-specific nodes into it when you first trained it).
Downside of that is the additional config required.

Adding by default - no, it's opaque behaviour IMO - the "normal logging" behaviour will disappear when users set any other listeners (i.e., they won't realize it's from a listener).
And there's no way to know an appropriate frequency for different nets (1ms per iteration vs. 10 seconds per iteration - huge spam vs. never reported, for same frequency). And disabling it is opaque (have to do setListeners(null) or something)

Yeah, I can see that.  For logging then, I would either use a separate interface or a subclass of of Listener, and do something like SameDiff.setLoggers() and addLoggers.

Also use validation results (if provided).

Not following

Probably better to add a listener method like validationDone, so that we could calculate metrics based on validation data instead of training data.

I like this. But I'm tempted to make this an enumeration:

Agreed
For the rest of the stuff, using the graph like that was mostly an attempt to avoid having to add separate Keras Metric type stuff, but I agree with you about the graph cleanliness stuff.
One issue is that for something like EarlyStopping, we need to be able to pass in a metric to watch.  Additionally, it needs to be able to be calculated for validation or training data.  Evaluation doesn't really work (although you could make Metric a subclass of it).  Ideally this would be able to access anything in the graph, not just inputs and results like Evaluation.
My inclination would be to do something like:
public interface Metric<T, R> {
    public T calculateForBatch(MultiDataSet data, SameDiff sameDiff, At at);

    public R aggregateBatches(List<T> batchResults);
}

public abstract class DoubleMetric implements Metric<Double, Double> {
    public AggregationMethod aggregationMethod(){
        return AggregationMethod.Average;
    }

    public Double aggregateBatches(List<Double> batchResults){
        return aggregationMethod().aggregate(batchResults);
    }
}

Metrics could be added by listeners (Listener.requiredMetrics()) or directly added passed to the SameDiff methods (I would probably have a common interface between them and IEvaluation so we could use one vararg param for both).
Listener methods could then receive a Map<String, Object> of metric results, and History would record all metrics and evaluations.
Also, unless I misread things, Evaluation doesn't have a defined score type method, so doing something like history.getEvaluation(Accuracy) would be impossible without getting the specific evaluation first.
Imo it might be better to keep Evaluation as a separate "evaluate after training" thing rather than using it in fit.  The only use I can think of would be to see something like a confusion matrix after every epoch.

Implemented long ago


Add hints to phone-number exercise

#220
Now that we have decided to only have hints for the first exercise using Option one could also opt that this one is not worth the effort?

I think these hints are fine. They are helpful and don't give away too much.

thx @ErikSchierboom


[function-mesh] POC of apko + melange to build python runner distroless image

fyi:
there was just a new release of apko v0.6 see https://github.com/chainguard-dev/apko/tags
in general, since apko and melange are busy projects with lots of changes but with lazy release process, one may think of working with latest code from main branch...

@hpvd Thank you~ I'll experiment with the new release~

fyi:
there is a good chance for a new release of melange at the end of december
see https://github.com/chainguard-dev/melange/issues/215#issuecomment-1356478888

@imaffe
the new release of melange (v0.2.0) is here:
whats new:
https://github.com/chainguard-dev/melange/blob/main/NEWS.md
release:
https://github.com/chainguard-dev/melange/releases/tag/v0.2.0

@hpvd I'm currently wrapping up the distroless process with Java runtime(which can be done using google/Distroless), for python runtime we will use melange + apko, and since Pulsar client now has a alpine release, I'll try if we can use apko only for python runtime


Update Streaming Support to match OpenAI streaming runs implementation

Streaming support for:
[ x ]  - retrieval
[ ] - function calling


Does the Kernel Instantiation node check the type arguments.

Can the Kernel Instantiation node throw?
It would throw if type bounds are checked at instantiation, and potentially every generic method would need different instantiation code.
@leafpetersen said:
Yeah, good question, this has been nagging at me.  The only situation in which there is an implied dynamic check on type arguments in a static invocation is when the bound of a type parameter itself involves a covariant type parameter from an outer class:
class A<T> {
  void foo<S extends T> (S x) {};
}

class B extends A<int> {
  void foo<S extends int> (S x) {x.isEven; }
}

A<Object> a = new B();
void Function(Object) foo = a.foo;  // Throws?
a.foo<Object>("hello"); // Throws
a.foo<Object>(3); // Still throws
foo("hello"); // Throws?

So the question is which of the two places should throw?
My assumption has been that implementations would put the check in the body of the method, and so the "convenient" place to throw is at the actual call site, not the instantiation site.  This is what DDC does.
It's probably slightly nicer to the user to throw at the instantiation site, but this is a pretty corner case example.
Thoughts from the implementations?  Things I'm missing? Lasse, Erik, thoughts?

We need a verdict from the language team before we can proceed.

My personal opinion is that
void Function(Object) foo = a.foo;

throws.
We are in a situation where a type argument to a class occurs contravariantly (the bound of type parameter is similar to the type of a parameter, it's supplied when the function is called or type-specialized), so we do need a runtime type check. I prefer to check early.
You are doing a tear-off of a method with static type void Function<S extends Object>(S) to a type of void Function(Object). The coercion is based on static types, so the method will be torn off and instantiated with Object as type argument. That's an invalid type argument for the actual function declaration, so the instantiation fails.
While we could delay until you call the method, it will just make the error harder to debug - you get a warning that a type argument is wrong at a position where there are not type arguments in the code.
This is also the most restrictive view (throws early), so we are not allowing something that can come back and bite us later.
So:
void Function(Object) foo = a.foo;  // Throws!
a.foo<Object>("hello"); // Throws!
a.foo<Object>(3); // Still throws!
foo("hello"); // Not valid if the tear-off threw.

@leafpetersen WDYT?

I argued that the dynamic error can occur at each invocation if we use the model where type arguments and value arguments are passed in one step. This corresponds to an approach where the tear-off operation produces a wrapper which will perform the complete invocation, passing both type arguments and value arguments:
  // Inference reveals that a generic function is assigned to a variable whose type
  // is a non-generic function type, hence we perform generic instantiation, guided statically.
  Function(Object) f = a.foo;
  // The tear-off works similarly to `(Object o) => a.foo<Object>(o)`, and each invocation will
  // incur a dynamic error when `a` has type `A<T>` for some `T` which is not a type type.

In contrast, we may also prefer to use a curried model (where it takes one step to pass type arguments and another step to pass value arguments):
  // Inference reveals that a generic function is assigned to a variable whose type
  // is a non-generic function type, hence we perform generic instantiation, guided statically.
  Function(Object) f = a.foo;
  // This amounts to `.. = a.foo<Object>`, which is an invocation of the instance method `foo` of
  // the given `a` where the type argument `Object` is passed --- so it fails here.

With the curried model I find it more difficult to justify a delayed error, it should happen at the location where we can meaningfully claim that the generic instantiation takes place.
That said, nobody seemed to have a particularly strong opinion about whether we should use the curried or the non-curried model. ;)

Can we specify that the location of the error may be chosen by the implementation?

Maybe it would be useful to know a bit more about the implications for the implementation. Is one choice hugely more desirable than the other?
(Of course, desirability has been considered from a language design perspective all the time, but if that doesn't provide a very obvious answer then maybe implementation considerations would step forward..)

Today the VM implements the "throw-at-call" behavior. This is more efficient that the "throw-at-instantiation" behavior given the current implementation of bounds checking. However, if we optimize bounds checks in the future, the "throw-at-instantiation" behavior may allow us to optimize more calls to generic functions which are also the target of an Instantiation.
Therefore, we would prefer if implementations are given the flexibility to throw the error at either location.

I believe that the usage of a type that covariantly depends on a type parameter from an enclosing entity (class or generic method) as the bound of a type parameter of a generic function will always be a weird corner case. Developers just shouldn't write code doing that (unless they are exclusively going to use that generic function in a scope where said type variable is still in scope), so we shouldn't be hugely friendly to them when they do that. ;-)
This basically means that the bounds on type arguments to generic functions should be compile-time constant types, rather than existential types for which it is only known that they have a certain upper bound (like S extends T in the declaration of foo in the initial text of this issue).
With existential open, the situation may change:
  A<T> a = ...;
  if (a is A<?X>) {
    // Here, `X` denotes the exact type argument of the
    // dynamic type of `a` at `A`.
    void Function<S extends X>(S) f = a.foo; // Safe.
    void Function(X) g = a.foo; // Instantiation with type argument `X`: Safe.
  }

But this would still work fine with both the curried and the uncurried model, it just happens to be safe so we don't have to worry so much about where to get the error. ;-)

Actually, the consistent approach to the generic instantiation of a function whose type argument bounds are existential would be to incur a compile-time error:
class A<T> { void foo<S extends T> (S x) {}; }
class B extends A<int> { void foo<S extends int> (S x) { x.isEven; }}
A<Object> a = new B();

 // Assume `S`. Compile-time error for all `S` except `Null`: not a subtype of `a.T`.
// If `Null` is inferred then it is a compile-time error (the downcast from
// `void Function(Null)` to `void Function(Object)` is statically known to fail, based
// on exact type information), or a dynamic error (when exact type info is ignored).
// The dynamic error will occur here because it's a plain (non-generic) function
// type subtype check that fails.
void Function(Object) foo = a.foo<S>;

// Error: infer `Null` (or raise error during inference), but then make it a compile-time
// error (using exactness) or a dynamic error (here!), as above.
void Function(Object) foo2 = a.foo;

// Compile-time errors: `Object` is not a subtype of the bound.
a.foo<Object>("hello");
a.foo<Object>(3);

// `foo` is null or has type `void Function(Object)`: Safe statically and
// dynamically to call it like this. But we won't reach this point, of course,
// even if the compile-time errors are eliminated, because the assignment
// to `foo` will be a dynamic error (when it isn't a compile-time error).
foo("hello");

The reason why it should be a compile-time error to pass a type argument which is not statically known to be appropriate is that this is what we do otherwise:
class A<X extends num> {}
class B<Y> {
  A foo() => new A<Y>(); // Error: `Y does not extend num`.
}

It is not impossible for Y to satisfy the bound, it's just possible that it doesn't, and that's an error.

@rakudrama @sigmundch Can you weigh in on implications for dart2js?
On usability grounds, I have a small preference for specifying that the check happens at the instantiation point (early).  But it's a very small preference, and if there are negative performance implications I'm happy to defer to those.
It seems to me that as currently specified, you always know statically whether or not a covariance check on the bound is required, since we only do these for tearoffs. In this case, I would expect performance/code size implications to be small since you only need to generate code to do the check for the rare cases that this actually happens (instead of for any possible universal function).
On the other hand, if we generalize this feature by adding explicit syntax for doing this (f<int>), then this would not be limited to tearoffs, and in that case every generic function would have to be prepared for this.  For methods, again, you know statically whether this is required or not, but for functions you won't.  So doing the check eagerly here feels like it could have a noticeable code size and potentially performance cost.
On the gripping hand, if we specify the check as happening early now, we can change our minds later in an essentially non-breaking way.
If the implementation folks basically agree with my analysis above, then I would say that lets go with an early (instantiation time) error for now, which allows us to move it later if we need to.
If there is a belief that this will incur a non pay-as-you-go code size/performance cost, then I don't think it's worth it, and we should go with the late error.
Whichever we decide, changing an existing implementation to conform seems fairly low priority to me relative to other pressing issues at the moment.

I think check-at-instantiation is the most usable version (avoiding re-checks on repeated calls, e.g. a sort predicate or fold-reducer).
It does mean larger code, since we have to be able to check some functions and not others. The pay-as-you-go cost is a special method on generic functions that require a run-time bounds check. I think we can make this pay-as-you-go in size at the runtime expense of a per-instantiation check for the presence of the bounds-check method.
Since dart2js will likely be used to ship code that has this kind of check disabled, lets go for the most usable version for the development.

Moving to area-language as it seems like we have not decided on the correct behaviour yet.
cc @leafpetersen @lrhn is this a blocker for stable?

For area-language on this topic, we have this CL and issue #33726. I suppose we should just mark this one as blocked on the latter and keep area-kernel (because we will then preserve the situation where there is an area-kernel issue, which will be needed anyway).

Done. I'd still like to hear how critical this change is?

The area-language decision that needs to be taken will make certain dynamic errors occur early (at tear-off) or late (at invocation), and the dynamic error will occur every time we have actual type arguments with values that violate the bounds (so we will have the error early, or we will have the error at every invocation of the resulting function object). This means that it only makes a difference with respect to "successful execution" in the case where a tear-off of this kind is never called.
I believe that this is such a subtle difference in practice that it can be considered non-critical. @lrhn, @leafpetersen?

My preference is to throw the error on instantiation.  It's non-breaking to change it from that to "on call" later if we want (but not vice versa).   @lrhn are you fine with that?
This is pretty corner case, so I think it's ok if we push this off of the Dart2Stable milestone and fix the implementations later.  There is a small risk though that there is code out there now which instantiates but never calls, and hence which will break when we fix this.

The feature specification generic-function-instantiation.md has been landed, and it includes the requirement that the above-mentioned dynamic error must occur at instantiation (that is, during tear-off). Hence removing the 'blocked' label.

Removing myself as an assignee, because the feature specification has been landed.

We have the wrong behavior for this in the VM, and need to fix it.

Closing this issue as the current behaviour matches the specification.


Adding the missing content on configuring keystores in migration from 2.x versions

Purpose
In API Manager 3.x versions, it is required to use a certificate with the RSA key size greater than 2048. When migrating from 2.x versions issues can be arisen if you do not configure your key stores properly.
Goals
Adding the missing documentation on correctly configuring key stores when migrating from 2.x versions to 3.x versions.
Approach

As shown in the above screenshot, the content was added to the missing documentation pages listed below.

APIM 2.2.0 to APIM 3.2.0
APIM 2.5.0 to APIM 3.2.0
APIM 2.6.0 to APIM 3.2.0

User stories

Users can configure the key stores properly when migrating and the issues will not arise.

Related PRs

List any other related PRs

@wasuradananjith  can we close this?


Fixed #27262 - Delegated URL resolver checks to URL classes

I cleaned up  check_resolver() to check for a contract (check(self)) instead of the patter's type.
By making all individual check_*() functions private members of the classes they were written to work with in the first place, the coupling is taken care of.
Finally, if I am not missing anyhing, the old code would only alert about startswith_slash if no other warning (specific to the class) was found (because of if not warning: which would't be empty otherwise). Now startswith_slash is checked by the base class of both patterns LocaleRegexProvider regardless of other warnings found.

buildbot, add to whitelist.

This is because, now, the first RegexURLResolver is checked and it fails as get_resolver(urlconf=None) returns RegexURLResolver(r'^/', urlconf), thus failing _check_pattern_startswith_slash()
One possible solution, would be to not check for the slash on RegexURLResolver.check() but it would mean children resolvers would not be checked either (when they used to be)

Would the semantics change if we were to replace the root URLResolver to use '^' instead of '^/'?
Or maybe we should special-case this specific pattern in "starts with slash", because it is not starting but ending with slash ;).

Changing the URLResolver returned by get_resolver to use '^' throws up some errors related to pickle and stuff of the sort, and it doesn't feel (to my inexperienced self) like the right way to go about it.
I feel that checking not regex_pattern.endswith('/') before warning is a much better idea.

Looks good to me.


Simplest way to setup / take photo with ip cam?

help?

It depends on model, but the simplest IMO is to call wget from command line:
$ wget "http://{ip.address}/snapshot.cgi?user={username}&pwd={password" -O photo.jpg

The {ip.address}, {username} and {password} need to be replaced by a real values, e.g.
$ wget "http://<IP_ADDRESS>/snapshot.cgi?user=admin&pwd=secret" -O photo.jpg

Writing program for this is not the simplest, however, if you insist to do that then take a look at the list of examples in the project main page, for example this one:
https://github.com/sarxos/webcam-capture/blob/master/webcam-capture-drivers/driver-ipcam/src/examples/java/MjpegLignanoBeachExample.java


dconf not found in PATH

I followed the directions for building and installing. I am running Xubuntu 19.04
$ autoplank dconf not found in PATH

The solution for me was to install dconf-tools from apt


Fix for flexbox container layout issues on older browsers

Confirmed affected browsers:
Internet Explorer 10
Safari on iOS 8.4 (iPad Air 2)
The container for the player poster did not have the Play button vertically aligned in the center.
Before:
https://www.dropbox.com/s/2hoae0yssccroc5/Screenshot 2016-08-08 14.14.27.png?dl=0
After:
https://www.dropbox.com/s/ujv85f01xi75hzt/Screenshot 2016-08-08 14.14.01.png?dl=0
When using flexbox with an older browser that implements an older version of the flexbox specification, prefixes must be added, in some cases, default values are different as well. See link:
https://msdn.microsoft.com/en-us/library/hh673531(v=vs.85).aspx
Prefixed values were generated with Autoprefixer: https://autoprefixer.github.io/

Coverage decreased (-0.02%) to 71.623% when pulling db694620f017a5327c9e300c85e05bd7fc218909 on michaelcunningham19:bugfix/poster-flexbox-prefixing into 1efbddf04a1aad4fc3d0aa7ebbfa94e90708b322 on clappr:master.

thanks @michaelcunningham19 :)
@clappr/core maybe we should use autoprefixer with webpack and have this be automatic?

@tjenkinson not really, we already use bourbon. We should use bourbon mixins, as we are already using on other plugins. This was only an oversight.
@michaelcunningham19 could you please update the PR using proper bourbon mixins instead of expanding the prefixes?

@towerz
I have updated the PR to use bourbon mixins 👍

Coverage remained the same at 71.644% when pulling 62c1754a1fd9a7bbbce2ac665aa0ac181ded43e8 on michaelcunningham19:bugfix/poster-flexbox-prefixing into 1efbddf04a1aad4fc3d0aa7ebbfa94e90708b322 on clappr:master.

Thanks, @michaelcunningham19! 🍻


Error gettting values when number is between quotes (string)

Hi,
I'm trying to parse a json object that contains:
  "NumberOfUsers":"30"
The number is between quotes. I can't change it, it an off the shelf app that generates this json.
In example.yaml I have the following metrics definition:
- name: NumberOfUsers
  description: Number of users
  path: $.NumberOfUsers

When I curl the endpoint it throws the following error:
2018-01-21 15:58:35,551 - root - DEBUG - metric_name: test_NumberOfUsers, value for '$.NumberOfUsers'
2018-01-21 15:58:35,551 - root - DEBUG - metric_name: test_NumberOfUsers, value for '$.NumberOfUsers' : 30
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/SocketServer.py", line 596, in process_request_thread
    self.finish_request(request, client_address)
  File "/usr/local/lib/python2.7/SocketServer.py", line 331, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "/usr/local/lib/python2.7/SocketServer.py", line 652, in __init__
    self.handle()
  File "/usr/local/lib/python2.7/BaseHTTPServer.py", line 340, in handle
    self.handle_one_request()
  File "/usr/local/lib/python2.7/BaseHTTPServer.py", line 328, in handle_one_request
    method()
  File "/usr/local/lib/python2.7/site-packages/prometheus_client/exposition.py", line 94, in do_GET
    output = generate_latest(registry)
  File "/usr/local/lib/python2.7/site-packages/prometheus_client/exposition.py", line 80, in generate_latest
    output.append('{0}{1} {2}\n'.format(name, labelstr, core._floatToGoString(value)))
  File "/usr/local/lib/python2.7/site-packages/prometheus_client/core.py", line 823, in _floatToGoString
    elif math.isnan(d):
TypeError: a float is required

I'm also hitting this issue

Hi, I also face this issue now


An issue

What if I dont want to use the editor when I play  and I have already installed  the among us editor. What do I do  so that I can use play without the editor even if it is  in my system and also because of some reason after i installed the editor I am not able to use the sabotage button and also after using the editor I am not getting enough public games to play.

This isn't the editors fault, you must've installed something else.
The editor doesn't "install", just try reinstalling your game.
If you have something like AmongUsPlus (Which isn't safe to use), then remove it.


Formatting a function removes formatting

Excel Labs 1.4

This is the expected behaviour as we reformat the formula to fit within the task pane. If you narrow the pane it will break.
We do not support specifying max line width is typical for editors - this is a feature gap.

Ah ok. In fact easy formatting was the main reason to test Excel Labs formula editor 😭. So I won't touch the formatting function for the time being
Is formatting planned in any way?


[PE-D][Tester C] App is inefficient in managing multiple classes

Currently, there is no way to segment students into classes for each event, and every student is assumed to be absent in every single class. If the TA manages multiple classes, then he/she would see students from other classes in an event, which can be very confusing.
The use of groups does not improve this as the listattn command cannot be filtered using groups.
A suggestion would be to implement a field known as class which allows TA's to specify which classes the students are in and listattn can filter by these classes.

Labels: severity.High type.FeatureFlaw
original: wang-h-z/ped#5

Added as planned enhancement number 2 in DG


使用搜索引擎问答，选择duckduckgo，开启科学上网，报错如下

RemoteProtocolError: peer closed connection without sending complete message body (incomplete chunked read)
Traceback:
File "/usr/local/lib/python3.10/dist-packages/streamlit/runtime/scriptrunner/script_runner.py", line 552, in _run_script
exec(code, module.dict)
File "/data/data/llm/code/Langchain-Chatchat/webui.py", line 57, in
pages[selected_page]"func"
File "/data/data/llm/code/Langchain-Chatchat/webui_pages/dialogue/dialogue.py", line 123, in dialogue_page
for d in api.search_engine_chat(prompt, search_engine, se_top_k):
File "/data/data/llm/code/Langchain-Chatchat/webui_pages/utils.py", line 213, in _httpx_stream2generator
for chunk in r.iter_text(None):
File "/usr/local/lib/python3.10/dist-packages/httpx/_models.py", line 844, in iter_text
for byte_content in self.iter_bytes():
File "/usr/local/lib/python3.10/dist-packages/httpx/_models.py", line 823, in iter_bytes
for raw_bytes in self.iter_raw():
File "/usr/local/lib/python3.10/dist-packages/httpx/_models.py", line 881, in iter_raw
for raw_stream_bytes in self.stream:
File "/usr/local/lib/python3.10/dist-packages/httpx/_client.py", line 123, in iter
for chunk in self._stream:
File "/usr/local/lib/python3.10/dist-packages/httpx/_transports/default.py", line 103, in iter
with map_httpcore_exceptions():
File "/usr/lib/python3.10/contextlib.py", line 153, in exit
self.gen.throw(typ, value, traceback)
File "/usr/local/lib/python3.10/dist-packages/httpx/_transports/default.py", line 77, in map_httpcore_exceptions
raise mapped_exc(message) from exc

请检查api与llm_api是否已启动，并检查是否使用默认端口7861与8888。

经检查发现是因为未安装duckduckgo的包，通过pip install duckduckgo-search解决

经检查发现是因为未安装duckduckgo的包，通过pip install duckduckgo-search解决

为什么我安装了也不行？


Upgrade to Java 11

Most systems now use java 11 as default & hence upgrade the build to use java 11

Maybe upgrade directly to Java 17.

Java 11 as a mininum requirement was taken care via #241 and hence modifying this ticket to go to Java 17/21.

@msmygit Can we close this issue as we had initially created this issue for Java 11?
We can create this once C*/DSE versions starts supporting JDK 17.


Reproducing MoCo on ImageNet-1k.

Hi Yonglong,
Thanks a lot for the great work and sharing the code. I am trying to reproduce the results of MoCo on ImageNet-1k, with ResNet 50. Did you reproduce the results on Kaiming's paper on the full ImageNet? Would you kindly share me the specific configurations for reproducing MoCo-ResNet-50?
Thanks a lot!

Hi, @yzou2,
Thanks for your kind words! I have reproduced his results  (60.6 with 64 negatives) on ImageNet. I strictly followed his pre-training recipe (8GPUs, 256 batch size, 200 epochs, blabla), and used a Distributed Dataparallel training code (I will release this once I got some time).

Thanks a lot, Yonglong!

Hi @HobbitLong ,
Did you modify other parts than this distributed data parallel training (will this distributed training influence accuracy)? Could you please tell that If I directly change the configuration to (8Gpus, 256 batch size, 200 epochs) without changing current code, would this achieve comparable performance?
Thanks!

Hi, @yzou2, @happywu,
Maybe this is a bit late, but just let you know that the DDP implementation for MoCo (as well as InstDis, MoCo v2, PIRL, ...) have been released here.

Hi, @yzou2, @happywu,
Maybe this is a bit late, but just let you know that the DDP implementation for MoCo (as well as InstDis, MoCo v2, PIRL, ...) have been released here.

It's really great! Thanks!


conda-env-activate does not run activate.d scripts

M-x conda-env-activate does seem to ignore contents of the conda environment's activate.d directory
For example, my q_activate.sh script  sets the QHOME environment variable and the test outside emacs works correctly:
$ echo $QHOME         # empty

$ conda activate q

(q) $ cat $CONDA_PREFIX/etc/conda/activate.d/q_activate.sh
#!/bin/bash
export OLD_QHOME="$QHOME"
export QHOME="${CONDA_PREFIX}/q"

(q) $ echo $QHOME          # set as expected
/opt/conda/envs/q/q

In emacs and with conda.el, when I activate the q environment, the QHOME env variable is empty:
M-x conda-env-activate
HELM Conda Env Activate -> Choose a conda environment: -> q
M-: (getenv "QHOME")
""

Albeit, PATH environment variable seems to be modified correctly by the activation:
M-: (getenv "PATH")
"/opt/conda/envs/q/bin:/opt/conda/bin:/opt/conda/condabin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games"

Expected Behavior
All scripts in $CONDA_PREFIX/etc/conda/activate.d and $CONDA_PREFIX/etc/conda/deactivate.d directories should be run, as per the conda docs.

@danielkrizian sorry for the delayed response, for some reason I haven't been seeing notifications for new issues. What would you like to see happen here -- more generally than just your specific use case? Since as you point out activate.d can contain arbitrary scripts, I'm not sure how we should run them and load any changed environment variables, settings, etc into Emacs!

In general use case, could conda-env-activate command also execute all *.sh scripts located in the activate.d directory of the selected conda environment? In any shell, running $ conda activate [selectedCondaEnvName] does it, conda.el currently does not.

@danielkrizian I'm still not clear on why you're asking for those scripts to be run, so I can't be sure if any of the possible solutions I'm thinking of will help you.
When you activate a conda environment in a shell, running all of the activate.d scripts updates the state of the shell -- which may be more than just environment variables. Will just updating the environment variables solve your problem? If so that's something I can look into.

Hi, I would also like to see this support. I have just started at a new team which is using conda, and it is all new to me so please excuse any terminology problems.
My team uses conda to install the gxx_linux-64 package which is the C++ compiler. On activation this runs the corresponding script. The main (if not only) purpose of this script is to set a bunch of environment variables which are used by the compiler, such as CC and CXX which point to the compiler itself, and also other environment variables which set compiler flags, linker flags, and more. These environment variables are needed when I build my project. The makefile uses $CXX to find the compiler and so it will fail if this isn't set. Right now when I use M-x compile it fails because this environment is not set.
In answer to your question I think that updating the environment variables will solve the problem (speaking for myself, not the OP).
For what it's worth, my workaround right now is to source the activation scripts prior to invoking the build tool when I use M-x compile. So for example, instead of using make -C {dir} {target} I use for s in $VIRTUAL_ENV/etc/conda/activation.d/*.sh ; do source $s ; done ; make -C {dir} {target}.

It seems like environment variables are the main use case for this, so
I think I've got a path forward. I'm not going to get to this soon,
unfortunately -- would appreciate any help, though! I'll try to have a
look this weekend.
On Wed, Apr 8, 2020 at 17:27, Alastair Rankine
<EMAIL_ADDRESS>wrote:

Hi, I would also like to see this support. I have just started at a
new team which is using conda, and it is all new to me so please
excuse any terminology problems.
My team uses conda to install the gxx_linux-64 package which is the
C++ compiler. On activation this runs the corresponding script. The
main (if not only) purpose of this script is to set a bunch of
environment variables which are used by the compiler, such as CC and
CXX which point to the compiler itself, and also other environment
variables which set compiler flags, linker flags, and more. These
environment variables are needed when I build my project. The
makefile uses $CXX to find the compiler and so it will fail if this
isn't set. Right now when I use M-x compile it fails because this
environment is not set.
In answer to your question I think that updating the environment
variables will solve the problem (speaking for myself, not the OP).
For what it's worth, my workaround right now is to source the
activation scripts prior to invoking the build tool when I use M-x
compile. So for example, instead of using make -C {dir} {target} I
use for s in $VIRTUAL_ENV/etc/conda/activation.d/*.sh ; do source $s
; done ; make -C {dir} {target}.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or unsubscribe.

Just adding a note that #53 was another vote for environment variable support.

Here is another vote in favor of environment variable support.
The problem I described in SciTools/cartopy#1725 also appears because of a script (proj4-activate.sh) which sets a new environment variable is not run on conda-env-activate.

I believe as of https://github.com/necaris/conda.el/commit/44df826ce92831c2f3b446778275194a51d388e7 this is solved using upstream functionality in conda itself. So I'm closing this, but if it's not actually fixed please reopen!

I have a activate.d script to set an env variable, but conda.el does not pick it up. Would love to know how I can add env variables on activating conda. (Running latest conda.el)


Redesign clock port

Refers to #12.
Used the shared event queue instead of threads.

@ARMmbed/mbed-os-core @ithinuel @ashok-rao

Just realized that I accidentally included the network PR into this one, closing and opening a new one.


MS: Create a new microservice MSOutput (simple functionality)

Impact of the new feature
ReqMgr2MS
Is your feature request related to a problem? Please describe.
This is part of the output data placement migration to WMCore, see:
https://github.com/dmwm/WMCore/wiki/ReqMgr2-MicroService-Output
There is the need for a new microservice to handle output data placement, which will initially work with PhEDEx and DDM, and in a late stage with Rucio.
Describe the solution you'd like
Re-use the MicroService architecture and add a new microservice to the MSManager module (and MicroService package).
It should have the very basic functionality, read the service configuration from the deploy configuration file, serve any kind of content through the info/status REST APIs. The skeleton has to consider different data management technologies.
Besides changes in the WMCore repo, we should also implement the necessary changes to the deployment repo (reqmgr2ms configuration).
Describe alternatives you've considered
none
Additional context
none

@amaltaro just to clarify. Here

The skeleton has to consider different data management technologies.
By 'data management technologies' you mean: Phedex, DDM and Rucio, right ?

That's correct!


MODLD-587: Authority validation - Creator of Work

new permission PR on UI side: https://github.com/folio-org/ui-ld-folio-wrapper/pull/10


docs: Add more detail around release process

Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:

[x] Make sure to open an issue as a bug/issue before writing your code!  That way we can discuss the change, evaluate designs, and agree on the general idea
[ ] Ensure the tests and linter pass
[ ] Code coverage does not decrease (if any source code was changed)
[x] Appropriate docs were updated (if necessary)

Fixes #778 🦕
I tried updating the README with the answers from #778. I hope this helps. Thanks so much for release-please!

@googlebot I signed it!

LGTM. Thanks!

@nzakas thank you for the contribution 👍


Add global function signature cache for default_key

Pull Request Test Coverage Report for Build 23

7 of 7 (100.0%) changed or added relevant lines in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.05%) to 96.91%

Totals

Change from base Build 22:
0.05%

Covered Lines:
345

Relevant Lines:
356

💛  - Coveralls


Carousel Component Not Responding To 'paginationPosition' Attribute Values

Bug Report
Current Behavior
Not responding to paginationPosition settings.
Input Code and steps to reproduce
Try to set the paginationPosition attribute on the carousel component.
Expected behavior/code
The valid values are top, bottom, top-overlay and bottom-overlay.
Setting top-overlay or bottom-overlay does not show any pagination dots. Not even when I make them red using the paginationColor attribute.
Setting either of the values top or bottom, always displays the dots at the bottom.
The dots have way too much top-margin from the slider. This should also be a setting.
Environment
All the latest. It's a vue-cli created app with the latest vue-carousel installed.

I'd also like the option of positioning the nav arrows. These are currently placed outside of the carousel. My carousel is 100% width. That means the nav arrows are placed outside of the carousel parent's container.
This might be considered a feature request, but I figured I'd mention it here, first.

okey i understand that can i try to resolve this issue


reset option

Reset option when starting firefly (delete AppData folder)
Who has forgotten the password can not log in otherwise

Hey @dev0pa this might not be a secure functionality to add. If the password to a profile is unknown, it is impossible to verify whether the user trying to delete the profile is the actual "owner" of that profile (i.e. an attacker might just try to delete the profile of a victim) in which case we do not want the user to be able to delete a profile from the outside.
But as you mention the data (for Windows) is in AppData, so it is still possible to manually / forcefully delete the profile data anyway.

Hey @dev0pa, moved this into a discussion here.


Problem with message_handler (aio)

The coroutine that this is embedded in is run with asyncio.run_coroutine_threadsafe() from another thread.
Any idea what I am doing wrong?
I set up the handler:
   bus = await MessageBus(bus_type=BusType.SYSTEM).connect()
   bus.add_message_handler(msg_handler)

The handler:
def msg_handler(msg):
    if msg.message_type is MessageType.ERROR:
        print(msg.error_name, msg.body)
        return True

Although I return True, the method call never returns.
        print('appkey_add call')
        await self.intf_node.call_add_app_key(self.path, dst_addr, app_idx, net_idx, False)
        print('appkey_add returned')

The output is as expected, just that the call never returns. The non-error case works as expected.
appkey_add call
org.bluez.mesh.Error.NotFound ['AppKey not found']

On exiting the program, I get:
Task was destroyed but it is pending!
task: <Task pending coro=<Menu.config_appkey_add() done, defined at ./prov_aio.py:1849> wait_for=<Future pending cb=[<TaskWakeupMethWrapper object at 0xffffb99ae1d0>()]> cb=[_chain_future.<locals>._call_set_state() at /usr/lib/python3.7/asyncio/futures.py:355]>

The problem isn't related to the handler. I removed it and I still get the hang on receiving an error response. Below is the dbus-monitor of correctly processed response and the error response that hangs.
method call time=1586794417.794161 sender=:1.754 -> destination=org.bluez.mesh serial=20 path=/org/bluez/mesh/nodef36118cde9ac47a3882be1d5537e7124; interface=org.bluez.mesh.Node1; member=AddAppKey
   object path "/mesh/cfgclient/ele00"
   uint16 1
   uint16 1
   uint16 0
   boolean false
method call time=1586794417.799934 sender=:1.605 -> destination=:1.754 serial=842 path=/mesh/cfgclient/ele00; interface=org.bluez.mesh.Element1; member=DevKeyMessageReceived
   uint16 1
   boolean false
   uint16 0
   array of bytes [
      80 03 00 00 10 00
   ]
method return time=1586794417.800147 sender=:1.605 -> destination=:1.754 serial=843 reply_serial=20
method call time=1586794422.527159 sender=:1.754 -> destination=org.bluez.mesh serial=21 path=/org/bluez/mesh/nodef36118cde9ac47a3882be1d5537e7124; interface=org.bluez.mesh.Node1; member=AddAppKey
   object path "/mesh/cfgclient/ele00"
   uint16 1
   uint16 3
   uint16 0
   boolean false
error time=1586794422.527706 sender=:1.605 -> destination=:1.754 error_name=org.bluez.mesh.Error.NotFound reply_serial=21
   string "AppKey not found"

What does the code you use to call the method look like?

I have a utility that talks to bluetoth-mesh. It uses cmd2 for the console commands and dbus-next for the communication. They both run in their own threads. The cmd2 stuff wants to be in the main thread. The dbus-next is in a separate thread. The dbus-next thread handles incoming messages. The cmd2 commands queue the coroutines in the dbus-next loop for specific commands with:
future = asyncio.run_coroutine_threadsafe(func(self, ns), main_loop)
``
where func() evaluates to the method that calls the problem code and ultimately ending up with the call to call_add_app_key that I posted above.

The case without the error return works.
This is my first venture into dbus-next, so I probably have missed something along the way.

Are you using the high level or the low level api to call the method? The high level throws exceptions and the low level does not.

@sbrown4 side note: you might have noticed that prompt-toolkit supports asyncio, so threads might not be needed

I'm using the high level call.
await self.intf_node.call_add_app_key(self.path, dst_addr, app_idx, net_idx, False)

But, no exception thrown.

Thanks for the hint. I don't need a custom handler to catch errors.
However, I sure can't find any place in the call reply notify chain that checks for MessageType.ERROR.
Is a _check_method_return() missing in aio's call()?

In the low level interface, errors are treated just as another kind of method return. The higher levels get to decide how to treat errors.
Are you catching DBusError in that path? You might be throwing that error and it's not being reported.
As for threading, the library isn't tested with threading so that might be an issue as well.
I should really add logging to see what's going on.

It really looks like aio's call() is missing its _check_method_return().
This experimental patch seems to fix it. Can you verify that this is correct?
--- a/dbus_next/aio/message_bus.py
+++ b/dbus_next/aio/message_bus.py
@@ -233,6 +233,7 @@ class MessageBus(BaseMessageBus):
         future = self._loop.create_future()

         def reply_handler(reply, err):
+            BaseMessageBus._check_method_return(reply, err, reply.signature)
             if err:
                 future.set_exception(err)
             else:

This causes test failures. I think I made a mistake in the docs because call() is not supposed to throw dbus error. The intent is really clear in the tests.
https://github.com/altdesktop/python-dbus-next/blob/018e7b77e59b334aa16324e485dd8926b171a924/test/test_aio_low_level.py#L32-L40
But if it fixes your issue, you may be onto something. If you can fix it without breaking the test, a patch is welcome.

The patch doesn't solve the problem. Ignore it.
If call_method is not supposed to raise an exception, I can use a custom message handler to field it.
However, the dbus error return message seems to hang things up such that the call_method doesn't return. I don't think this should occur on an unrecognizable return message. I'll see if I can understand what's going on.

The issue appears to be in aio ProxyInterface._add_method.method_fn. The call_method return does get checked for MessageType.ERROR in BaseProxyInterface._check_method_return. A DBusError gets raised. I didn't have the call_method in a try block and missed the exception. I don't know why the exception output got suppressed without the try.
It appears call_method functions as currently documented.
All in all, sorry for the noise.

Further on the above's missing coroutine exception output.
It appears that a coroutine submitted from another thread by
f = asyncio.run_coroutine_threadsafe()

suppresses exceptions. They are only exposed if there if a
result=f.result()

follows. Then the exception is raised. The traceback can be retrieved with something like
try:
    f = future.result()
except Exception as e:
    print("".join(traceback.TracebackException.from_exception(e).format()))