DropDownField hintText is not aligned properly when the items are empty

Is there an existing issue for this?

[X] I have searched the existing issues
[X] I have read the guide to filing a bug

Steps to reproduce
run this dartpad to see the broken alignment.
Expected results
The hint text should be align properly even if the item's list is empty.
Actual results
The hint text is not aligned properly when the item's list is empty.
Code sample
Code sample
import 'package:flutter/material.dart';
void main() {
  runApp(const MaterialApp(
    title: 'Flutter Demo',
    home: MyHomePage(),
  ));
}

class MyHomePage extends StatefulWidget {
  const MyHomePage({super.key});

  @override
  State<MyHomePage> createState() => _MyHomePageState();
}

class _MyHomePageState extends State<MyHomePage> {
  @override
  Widget build(BuildContext context) {
    return Scaffold(
      body: Center(
        child: Padding(
          padding: const EdgeInsets.all(50),
          child: Column(
            mainAxisAlignment: MainAxisAlignment.center,
            children: [
              DropdownButtonFormField(
                decoration: InputDecoration(
                  hintText: 'Hint Text',
                  border: OutlineInputBorder(
                    borderRadius: BorderRadius.circular(14),
                  ),
                ),
                items: []
                    .map(
                      (e) => DropdownMenuItem(
                        value: e,
                        child: Text(e),
                      ),
                    )
                    .toList(),
                onChanged: (_) {},
              ),
              const SizedBox(height: 30),
              DropdownButtonFormField(
                decoration: InputDecoration(
                  hintText: 'Hint Text',
                  border: OutlineInputBorder(
                    borderRadius: BorderRadius.circular(14),
                  ),
                ),
                items: ['asd']
                    .map(
                      (e) => DropdownMenuItem(
                        value: e,
                        child: Text(e),
                      ),
                    )
                    .toList(),
                onChanged: (_) {},
              ),
            ],
          ),
        ),
      ),
    );
  }
}

Screenshots or Video

Screenshots / Video demonstration

Logs
No response
Flutter Doctor output
Doctor output
[✓] Flutter (Channel stable, 3.13.6, on macOS 13.5.2 22G91 darwin-arm64, locale
    en-EG)
[✓] Android toolchain - develop for Android devices (Android SDK version 33.0.2)
[✓] Xcode - develop for iOS and macOS (Xcode 14.3.1)
[✓] Chrome - develop for the web
[✓] VS Code (version 1.82.2)
[✓] Connected device (2 available)
[✓] Network resources

Hi @hasanmhallak. Thanks for filing this. I see there is an open issue describing this case https://github.com/flutter/flutter/issues/111958. Kindly comment, upvote and follow up on this issue for updates. Closing this in favour of the already existing issue.
Thank you


Readd 2.1 CORS functional tests

We lost CORS FunctionalTests when changing 2.1 to ProjectRef.

Nvrm, realized project structure changed.


New Line not working in Dialog Object in v15

It seems that with v15 the ability to do a new line within a Dialog object has changed or been broken.
The following code has been written and deployed against a Production environment running v14 and a Sandbox environment running v15.1:

The outcomes of this code differ based on the version it is run on.
V14

V15.1

Thank you for reporting that. We already know about that bug and working on it.
Be aware that bugs should be submitted via proper channels:
On premises - https://support.microsoft.com/en-us/supportforbusiness/productselection?sapId=93d37907-ad94-d591-22e9-593cfa09dd3f
On line – contact your partner, or if you are a partner open a support incident through Partner Center - https://partner.microsoft.com/en-us/pcv/dashboard/overview


Request Safe info from a Safe app

When the SDK is initialized it requests the Safe web interface the information of the Safe.
There is no need to export this functionality because is enough if it's called during the initialization.
This allows to initialize the SDK some time after the app is loaded and the Safe web interface sends the Safe info for the first time.
Check https://github.com/gnosis/safe-react/pull/1138 to see the changes from the Safe web interface side

The thing is that if the CPK or safe-apps-sdk are not instantiated by the time the Safe web3 interface sends the Safe info message to the Safe app, the Safe app won't receive it.
For example, when an app using a "login" button to instantiate the CPK/safe-apps-sdk.


JQ-JSON to Haystack Doc

Related Issues

fixes #issue-number

Proposed Changes:
User provides a JQ query and a JSON file.  The file can be in standard or JSON lines format. The output of the query and accompanying metadata are embedded in a Haystack Documents object
How did you test it?
Unit tests for testing as a standalone component and used a driver to test it in a pipeline
Notes for the reviewer

Checklist

I have read the contributors guidelines and the code of conduct
I have updated the related issue with new insights and changes
I added unit tests and updated the docstrings
I've used one of the conventional commit types for my PR title: fix:, feat:, build:, chore:, ci:, docs:, style:, refactor:, perf:, test:.
I documented my code
I ran pre-commit hooks and fixed any issue

Had the wrong commit


Swedish characters åäö not replaced correctly with urlencode()

Hi,
I am using your urlencode-function, but found it did not handle Swedish characters åäö correctly.
My workaround was simple:
  // åäö is not replaced correctly. 'å' for example, encodeURIComponent() produce "%C3%A5", while php's urldecode() expects "%25C3%25A5".
  str = str.replace(/å/g, "%25C3%25A5");
  str = str.replace(/ä/g, "%25C3%25A4");
  str = str.replace(/ö/g, "%25C3%25B6");

http://locutus.io/php/url/urlencode/

"%C3%A5" for å is correct. "%25C3%25A5" would be double-encoded ("%25" is the sequence for the percentage sign "%").

Interesting thought! It was many years ago and I had a database with things other people had been working with, so it is probably as you say, I was decoding twice.


keymanager: support private key rotationPolicy field

What this PR does / why we need it:
Builds on top of #2812 and adds support for a new certificate.spec.privateKey.rotationPolicy field.
This allows controlling whether private keys are rotated upon re-issuance or re-used.
For backward compatibility, the empty value is treated as 'Never' which matches the behaviour we have today. In a future API version, we can flip this default to be Always.
Which issue this PR fixes: fixes #2402
Special notes for your reviewer:
I'm going to hold this for now whilst I write unit tests 😄
Release note:
Add support for private key rotation when renewing or re-issuing certificates. This feature requires use of the new 'experimental' certificates controller. Set `certificate.spec.privateKey.rotationPolicy` to `Always` to enable this functionality.

/hold

/test pull-cert-manager-experimental-e2e-v1-17

/test pull-cert-manager-experimental-e2e-v1-17

/test pull-cert-manager-experimental-e2e-v1-17

/kind feature

Ready for review 😄
/hold cancel
/test pull-cert-manager-experimental-e2e-v1-17
/assign @meyskens

/test pull-cert-manager-experimental-e2e-v1-17

Updated @meyskens!

/lgtm


PseudoDynamic.Terraform.Toolset

Hey, as the "Discussions"-tab is not yet enabled, I hope it is okay to contact you through an issue. 🙂
I wanted to ask you, whether you are interested in developing a more mature Terraform Plugin SDK. I started here where I have already implemented a Provider Schema API represented by POCO classes and attributes to skip the need to first understand the MessagePack protocol and Plugin protocol in detail before you are able to write schemas. By introducing a separate layer on top of MessagePack it also enables you to provide custom types, for example ITerraformValue, that allows you to get the information whether the int value is unknown or not, similiar to github.com/hashicorp/terraform-plugin-go/types and their "ITerraformValue<>"-representation for every type, for example types.Bool or types.String.
If you are interested you can contact me easily with the email in my GitHub profile. 🙂

Yes, you are right. I think that this state is for a little bit longer. But don't get me wrong: There is always a way too participate and foremost I look for a core team that wants to build up a good API. When a project exists for decades, I normally expect this, examples and such, but like I just said, looking right now more into core developers that have spend some bare time on a side project.

@ThomasBleijendaal you may now take a look again. A lot has been done. Maybe @SamuelFisher is interested too in participating?
README → https://github.com/pseudo-dynamic/PseudoDynamic.Terraform.Toolset
Example → https://github.com/pseudo-dynamic/Terraform.Provider.Scaffolding


Add a display option that disregards file structure

We we run jest from the command line, it displays the tests as they are logically structured, not as they are distributed on disk. I wish JTE had a display option similar to that.
I don't want to see a tree that reflects file organization. I want to see a tree composed of describe and it/test blocks, without any visual indication of the file structure.

Came here to say the same thing. I'm looking at the code now and it seems like a pretty straightforward thing to add, so I'll try to get a PR up shortly.

Hi @tomprogers / @dannymcgee,
Question for you both: can you look at the following page on the host extension site and tell me if it meets your needs?
https://github.com/hbenl/vscode-test-explorer
The setting testExplorer.mergeSuites

@rossknudsen Unfortunately not for me. I do have that option enabled and I can see why it's useful, but the pain point I'm trying to address is that my test files are all pretty deeply nested inside my workspace directory structure, so I need to expand 5 or 6 nodes in the UI to see my first actual test suite, which leads to a lot of wasted horizontal space in the sidebar and a lot of clicking to see anything. If you take a look at the screenshots in my PR, the issue I'm having should be more clear.

nah, that's fine.  I just thought I'd check before merging.  I'll have another look later on, but the initial review looked good.

Heya! Thanks for following up.
I think I'm with danny on this. I do use the mergeSuites setting, but it's still not enough. As an example, I'm working on an Electron app, which adds another layer of folders to accommodate the main and renderer threads. By the time an actual describe appears, it's already indented like 4 levels. When I run jest, I don't see any of that wrapping FS structure.


Invoke (websocket) doesn't support multiple returns

When using the web-socket client, the first node that returns an output (message type =  executed) terminates the connection:
https://github.com/StableCanvas/comfyui-client/blob/32aba55d5dd6890b5f1650484b084c2928fd9768/src/InvokedWorkflow.ts#L144-L152
But looking at the Comfy UI message types, there wasn't an obvious message to listen for show that all nodes have completed other than perhaps watching the queue count on the status message.
The immediate workaround is to just use the invoke_polling method, which works fine for my purposes. But thought I'd get your thoughts before trying to fix it in my fork.

Fixed https://github.com/StableCanvas/comfyui-client/commit/8a7dc42a58c2d446d400d713c1228290799dec87
I discovered another issue, so I fixed it together. You can try it out, and if there are still problems, we can continue discussing.


Detect PDFs with bad censorship attempts (e.g. black squares that can be removed to obtain the "hidden" information)

https://github.com/freelawproject/x-ray


Optimize the AMD GPU Gemm parameterization backend

Does this need rebased on top of master or the commits squashed? That seems like a lot of commits to add to the history.

@CodeplayBen I have merged master on it, but not rebased (rebasing would be very painful here). The commits come from the tsgemm branch, squashing should be fine, like we always do


Allow token invalidation

Hello @jpadilla,
I was having discussion with another dev and thinking about the ember jwt app and the potential to add an enhancement that allows activity based token expiration.
At a high level the frontend would authenticate as usual but begin a countdown timer after last activity of user. If the countdown got to 0 it would issue an api request to invalidate the token on the backend and log them out on the frontend.
Just wanted to see what your thoughts are on this? Is it a logical use case? I know i've seen sites (banking etc) that log you out after inactivity....
-E

Hm after thinking about this some more maybe its not as straightforward as I was thinking... It seems like there is no obvious way to invalidate a jwt token? I found this SO link that touches on it.

@erichonkanen shouldn't your first option here be short expire times?

That depends.. if you don't mind having large numbers of token refresh requests then sure that would work for ensuring that a logged out client's key would be invalidated within a short time. If you didn't want to use such short expire times there is still an issue that it doesn't seem there is any straightforward way to invalidate a single jwt token.
Not sure if an activity-based model would work with jwt

closing as unsure of use case and ability to invalidate a jwt token


Examples of non-generative usage

Hi,
I would like to kindly ask you to provide some pointers/examples on using 4M for such tasks as retrieval and classification. How do you load only the encoder, for instance?

Hi @slowwavesleep ,
Thanks for your interest! I pushed some changes to make it easier to load pre-trained 4M checkpoints as a ViT backbone. Here's an example of loading 4M-21 B weights as a ViT backbone, and adding a simplified 1000-way classification head on top of it:
import torch.nn as nn
from einops.layers.torch import Reduce
from fourm.utils import load_safetensors
from fourm.models.fm_vit import FMViT

device = 'cuda' if torch.cuda.is_available() else 'cpu'

ckpt, config = load_safetensors('./4M-21_B.safetensors')

cls_head = nn.Sequential(
    Reduce('b n d -> b d', 'mean'),
    nn.LayerNorm(config['dim'], eps=1e-6),
    nn.Linear(config['dim'], 1000),
)

fmvit = FMViT(config, output_head=cls_head).to(device)
msg = fmvit.load_state_dict(ckpt, strict=False)
print(msg) # Check that the only missing keys are from the optional output_head

# Example forward pass. Input images should be ImageNet-standardized.
logits = fmvit(torch.randn(2,3,224,224).to(device)) # Returns (B, 1000)

For retrieval, we are actually performing generation in the same manner as for all the other modalities. Given an arbitrary input conditioning, we can predict either the tokenized global DINOv2 embeddings, or the tokenized global ImageBind embeddings. These need to be decoded using their respective tokenizer decoders. Retrieval is performed in a standard manner, i.e. by retrieving the sample with the lowest cosine distance to the predicted query embedding.
Best, Roman

Thank you, @roman-bachmann!

For retrieval, we are actually performing generation in the same manner as for all the other modalities. Given an arbitrary input conditioning, we can predict either the tokenized global DINOv2 embeddings, or the tokenized global ImageBind embeddings.

Oh, I see. That makes sense now. I initially thought you were just taking the encoder's outputs, but it's actually a new paradigm (at least to me). So if you treat the embeddings as modalities, then it should be possible to convert any model's output to any other model's outputs with 4M and some additional training? Like RoBERTa to GPT-2, for instance (disregard the lack of practicality in this example).
Could you guesstimate the difference in inference time, when using an image to produce DINOv2 embeddings with DINOv2 directly VS. using let's say text input to generate DINOv2 embeddings with 4M? Is it like an order of magnitude or less?

Oh, I see. That makes sense now. I initially thought you were just taking the encoder's outputs, but it's actually a new paradigm (at least to me). So if you treat the embeddings as modalities, then it should be possible to convert any model's output to any other model's outputs with 4M and some additional training? Like RoBERTa to GPT-2, for instance (disregard the lack of practicality in this example).

Exactly! Any output from another model or any sensor can be turned into a modality if you can find a way to turn it into discrete tokens. We view modalities as any signal that describes some factor(s) of the underlying reality of the scene. This is intentionally quite vague, but it means one can include real sensory measurements (e.g. RGB), semantics and subjective expression (e.g. captions), pseudo labels from specialist networks (e.g. depth maps, bounding boxes, human poses, etc), neural network features (e.g. dense feature maps or global embeddings), etc...
If you have an aligned dataset of those modalities, after training with the multimodal masked modeling objective, any subset of modalities can be mapped to any other, in a fully generative manner.

Could you guesstimate the difference in inference time, when using an image to produce DINOv2 embeddings with DINOv2 directly VS. using let's say text input to generate DINOv2 embeddings with 4M? Is it like an order of magnitude or less?

It's around the same order of magnitude if you do a single forward pass (assuming the networks are of similar sizes). We've observed that one can perform pretty decent text-to-image retrieval when using 4M to generate DINOv2 embeddings, even when predicting the 16 global tokens in a single forward pass (as opposed to generating them in multiple steps using the MaskGIT/ROAR decoding scheme).
Unlike when predicting DINOv2 embeddings from images (which is a somewhat deterministic setting), text-to-DINOv2-embedding prediction is underspecified since a given caption can map to an entire distribution of DINOv2 embeddings which match that caption. It's interesting that it works quite well with a single generation step -- something to look into further in future work.

I also wanted to confirm my understanding of your example with loading 4M as ViT backbone. The loaded model is still the cross-modal one, so in principle (but not in this specific example, I guess) it can be used to classify arbitrary combinations of modalities as inputs. Does that sound about right? One usage example I can think of is something like visual NLI, where you classify text/image pairs. 4M would seem like a suitable choice for that then.
Another thing I didn't fully understand from the papers is tokenization. If you tokenize all modalities into discrete tokens, then the model should have a fixed vocabulary then? Or the tokenizers actually output token vectors that go straight into the model? I'm not sure how that works with text then.

I also wanted to confirm my understanding of your example with loading 4M as ViT backbone. The loaded model is still the cross-modal one, so in principle (but not in this specific example, I guess) it can be used to classify arbitrary combinations of modalities as inputs. Does that sound about right? One usage example I can think of is something like visual NLI, where you classify text/image pairs. 4M would seem like a suitable choice for that then.

For simplicity we made FMViT specific to RGB pixel inputs to serve as a ViT backbone, but you are right that the underlying model that is loaded is still the cross-modal one. It should be relatively simple to extend FMViT to multimodal inputs by adding back the other modality embeddings (as in the FM class).

Another thing I didn't fully understand from the papers is tokenization. If you tokenize all modalities into discrete tokens, then the model should have a fixed vocabulary then? Or the tokenizers actually output token vectors that go straight into the model? I'm not sure how that works with text then.

Each modality has a fixed vocabulary size, yes. The 4M model has learned embedding layers for each modality which transform the discrete codes into vectors. The only exceptions to that are RGB pixels and T5 embeddings, which are input-only and are fed into 4M using learned linear projections (patch-wise for RGB).

Thanks again for the discussion and awesome work, @roman-bachmann!

I also wanted to confirm my understanding of your example with loading 4M as ViT backbone. The loaded model is still the cross-modal one, so in principle (but not in this specific example, I guess) it can be used to classify arbitrary combinations of modalities as inputs. Does that sound about right? One usage example I can think of is something like visual NLI, where you classify text/image pairs. 4M would seem like a suitable choice for that then.

For simplicity we made FMViT specific to RGB pixel inputs to serve as a ViT backbone, but you are right that the underlying model that is loaded is still the cross-modal one. It should be relatively simple to extend FMViT to multimodal inputs by adding back the other modality embeddings (as in the FM class).

Another thing I didn't fully understand from the papers is tokenization. If you tokenize all modalities into discrete tokens, then the model should have a fixed vocabulary then? Or the tokenizers actually output token vectors that go straight into the model? I'm not sure how that works with text then.

Each modality has a fixed vocabulary size, yes. The 4M model has learned embedding layers for each modality which transform the discrete codes into vectors. The only exceptions to that are RGB pixels and T5 embeddings, which are input-only and are fed into 4M using learned linear projections (patch-wise for RGB).

However, I noticed that any modality can generate RGB pixels in Fig.2 of the original 4M-21 paper. Does that mean the RGB pixels are not input-only?

Hello @roman-bachmann
Firstly, thank you so much for the amazing work! I just want to also expand on the following for using 4M-21 for retrieval purpose.

For retrieval, we are actually performing generation in the same manner as for all the other modalities. Given an arbitrary input conditioning, we can predict either the tokenized global DINOv2 embeddings, or the tokenized global ImageBind embeddings. These need to be decoded using their respective tokenizer decoders. Retrieval is performed in a standard manner, i.e. by retrieving the sample with the lowest cosine distance to the predicted query embedding.

To achieve functionality as shown in fig.5 of the paper (also shared below):

Is this what we do:

Use input modalities such as Query Image + Text (or subset of 21 input modalities) to generate an image
Use the generated image and use DinoV2 global embedding tokenizer to convert the generated image to tokens
Use the DinoV2 global embedding tokenizer to also convert a number of images to tokens (this is our database that we want to do retrieval from)
Finally perform a cosine similarity to get top-N retrieved results

Does the above summary seem correct?

Hi @amaarora ,
You don't have to predict an image as an intermediate stage. You can directly predict DINOv2 (or ImageBind) global embeddings from any subset of modalities. Then, given the global embeddings of the query and retrieval set, you can perform retrieval by comparing their cosine similarities.
So, a summary would be as follows:

Load DINOv2 (or ImageBind) global embedding tokenizer.

tok_dinov2_global = VQVAE.from_pretrained('EPFL-VILAB/4M_tokenizers_DINOv2-B14-global_8k_16_224')

Use any input modality to predict DINOv2 (or ImageBind) global embeddings. This will be your query embedding.

cond_domains = [COND_DOMAIN1, COND_DOMAIN2]
target_domains = ['tok_dinov2_global']
tokens_per_target = [16]
autoregression_schemes = ['autoregressive']

Obtain the DINOv2 global embeddings for the retrieval set in a similar way. If you are using RGB images as retrieval set, alternatively, first predict DINOv2 global embedding of them using the pretrained DINOv2 (or ImageBind) model, then tokenize the global embeddings using the tokenizer from Step 1.

Perform cosine similarity comparison between the query and retrieval set.

Hey @ofkar, thank you so much! This is very helpful.
I am writing a blog post on this and will share a draft shortly. :)


Function Tags

This PR contains the following changes:

Function tag utility functions are now refactored into a single file. This make it easier to manage them as we don't have to look in multiple files (UTF_Basics and UTF_Utils).
f632d98 (Refactor: Move function tags functions to own file, 2023-01-16)
Very long function names get now a short version when using them as dimension label to store function tags.
fc2f76a (Function Tags: Fix long function name bug, 2023-01-16)
Function tags are now sanitized to fix new dimension label behavior in Igor 9.
e3c9940 (Function Tags: Cleanup function tags, 2023-01-16)

Close #378
Close #328

@t-b @MichaelHuth Ready for Review.

Review:
f632d98c (Refactor: Move function tags functions to own file, 2023-01-16)
Looks good.
fc2f76ac (Function Tags: Fix long function name bug, 2023-01-16)
The approach here with the quick and long path seems quite complicated. Why not just use a 1D textwave for storing the full function names?
e3c99406 (Function Tags: Cleanup function tags, 2023-01-16)
I think CleanupName could replace SanitizeName or?

The representation of the function tag itself is kept to keep compability with older experiments.

We don't need that. In case it would your live easier.
Tests are missing for both fixes.

Another comment:
In GetFunctionTagWave the line
tag_constants[][1] = SanitizeTagName(tag_constants[p][0])

will result in the cleaned up name being found. But I don't think this is what we want.

@t-b

The approach here with the quick and long path seems quite complicated. Why not just use a 1D textwave for storing the full function names?

Do you want to throw dimension labels away at all and use FindValue in one wave (text wave) to find the index which is at the same time the same index in the second wave (wave reference wave)? My fix tried to keep the old behavior as much as possible and used the part with the dimension labels to its limits.

We don't need that. In case it would your live easier.

That's good. It is easier to change the tag to a clean one in the first place than the part with the cleanup. So I will change this completely.

Do you want to throw dimension labels away at all and use FindValue in one wave (text wave) to find the index which is at the same time the same index in the second wave (wave reference wave)? My fix tried to keep the old behavior as much as possible and used the part with the dimension labels to its limits.

Yes that would be my favourite.

@t-b @MichaelHuth Ready for next review round.


ASP.NET Core release page link not working

The "ASP.NET Core release page" link on https://github.com/dotnet/core/blob/master/release-notes/1.0/1.0.5.md is not working. It is pointing to https://github.com/aspnet/home/releases/1.0.5 and gives a 404.

@leecow @danroth27 1.0.5 does not exist on ASP.NET page: https://github.com/aspnet/home/releases
Is that expected?

@danroth27 , I think you need to get a 1.0.5 release page up. Is that correct?

Looks like the ASP.NET Core 1.0.5 release tag is live.


Red tests

I noticed that there are 17 tests which are failing both locally and in TravisCI. Would you consider removing or muting those tests since they don't provide any value? I could help to ✂  or mute them :)
P.S. ~TravisCI didn't trigger for this PR https://github.com/ankhers/mongodb_ecto/pull/167 for some reason.~ Force-push triggered the build

I am currently not interested in removing or muting them. The current 2.x branch will no longer be worked on. I believe I will be started from scratch when starting Ecto 3.x support.


slowdown after some time

Noticed that after about 80 animation frames, the display updates slower and several generations are dropped...
is this a known problem with gloom´s animate function?

No, it's actually a problem of the implementation, which never drops any information about older generations. As a result, memory consumption explodes and this reveals itself as slowdown.
So Gloss is pretty fine, the problem is in my code :(

ok... didn't try it yet, but I guess that gameOfLifeSuperSlow will not show this behavior ?

It will also run slower each time, but for a different reason: each round it needs to recompute all the previous generations.


added lots of ios-13-2-dep-9-3 toolchains (individual devices, signed and unsigned)

Hi, added new toolchain without bitcode. Thx, clemens

and more toolchains...

Awesome, thx. Just a quick question: did you ever encounter that running a
build for all devices, say armv7, armv7s and arm64, fails, while individual
ones work out perfectly?
I ran the “device” toolchain, for which I get a compiler error. But running
toolchains individually work out. It is not a memory issue, and I have no
idea...
Ruslan Baratov<EMAIL_ADDRESS>schrieb am So. 16. Feb. 2020 um
21:50:

Merged #341 https://github.com/ruslo/polly/pull/341 into master.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/ruslo/polly/pull/341?email_source=notifications&email_token=AAA3R4ECU7TQMMIKPJBZ5V3RDGRJFA5CNFSM4KUOFKJKYY3PNVWWK3TUL52HS4DFWZEXG43VMVCXMZLOORHG65DJMZUWGYLUNFXW5KTDN5WW2ZLOORPWSZGOWVHLJ5I#event-3041834229,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAA3R4FAGR7U6BUY7DFZNNDRDGRJFANCNFSM4KUOFKJA
.

did you ever encounter that running a
build for all devices, say armv7, armv7s and arm64, fails, while individual
ones work out perfectly?

Yes, it happens sometimes. Not all CMake code is suitable for a multi architecture build, not all C++ code as well. E.g. try_compile will run only for a single architecture, and if the results of this operation will be architecture-dependent, you will have errors while compiling the untested architecture. Same problem if you save the results generated by configure_file C++ source.
Also, note that for a single architecture build we will set CMAKE_SYSTEM_PROCESSOR, for the multi architecture we can't:

https://github.com/ruslo/polly/blob/99c7b708fed6a17488e11ca087a2beabf42e4dad/os/iphone.cmake#L157-L165

Examples:

https://github.com/hunter-packages/curl/commit/72243ea2dc71629422d9d693e5b8ee8f5afd2a12
https://github.com/hunter-packages/curl/commit/4ff2384b47719e2e8de4dc82cdf5d28e36149952
https://github.com/hunter-packages/opencv/issues/28


Tensorflow version of SCNN is available now

I have implemented SCNN using Tensorflow and put the full codes here. You can test the code in popular lane detection benchmarks like TuSimple, CULane and BDD100K or your custom dataset with minor modification. Welcome to raising issues if you can not reproduce the results. @ytzhao , @ding-hai-tao, @ycszen, @FrancisGee, @hewumars

Tensorflow version of SCNN版本是基于Lanenet网络方法写的吗 还是基于之前的Spatial CNN for Traffic Lane Detection，我下载了Tensorflow version of SCNN代码感觉和之前的Lanenet检测方法很类似。

@yinhai86924, Tensorflow version of SCNN就是在Lanenet的代码基础上改写得到的，主要不同的地方是：1. 网络结构中加入了message passing模块，同时用spatial upsampling替代了基于FCN的decoder模块；2.多输出，包括multi-channel的概率图输出和对于每条lane的二值预测；3.存储概率图的路径是按照SCNN来的，这样便于后面的测试。

Hi. I copy your full SCNN tensorflow code and CULanes datasets. But the loss is very small from epoch 1(shown bellown).However, the model for test runs OK. So I'm confused that the model is trainged from your repo code?  The lane mask is a small part in one image, so I doubt the softmax would't works fine for this situation?
Epoch: 1 loss_ins= 0.637915 (0.637915) loss_ext= 0.696006 (0.696006) accuracy= 0.178814 (0.178814) accuracy_back= 0.235519 (0.235519) mean_time= 50.792227s
Epoch: 2 loss_ins= 0.341373 (0.341373) loss_ext= 0.670495 (0.670495) accuracy= 0.021013 (0.021013) accuracy_back= 0.889111 (0.889111) mean_time= 0.332169s
Epoch: 3 loss_ins= 0.167313 (0.254343) loss_ext= 0.682850 (0.676673) accuracy= 0.000000 (0.010506) accuracy_back= 0.998962 (0.944036) mean_time= 0.328959s
Epoch: 4 loss_ins= 0.165585 (0.224757) loss_ext= 0.794752 (0.716033) accuracy= 0.000000 (0.007004) accuracy_back= 0.999992 (0.962688) mean_time= 0.335514s
Epoch: 5 loss_ins= 0.320033 (0.248576) loss_ext= 0.667665 (0.703941) accuracy= 0.000000 (0.005253) accuracy_back= 1.000000 (0.972016) mean_time= 0.335776s
Epoch: 6 loss_ins= 0.311843 (0.261229) loss_ext= 0.636154 (0.690383) accuracy= 0.000000 (0.004203) accuracy_back= 1.000000 (0.977613) mean_time= 0.335177s
Epoch: 7 loss_ins= 0.259280 (0.260904) loss_ext= 0.627233 (0.679858) accuracy= 0.000000 (0.003502) accuracy_back= 1.000000 (0.981344) mean_time= 0.334006s
Epoch: 8 loss_ins= 0.173998 (0.248489) loss_ext= 0.688290 (0.681063) accuracy= 0.000000 (0.003002) accuracy_back= 1.000000 (0.984009) mean_time= 0.334256s
Epoch: 9 loss_ins= 0.178677 (0.239763) loss_ext= 0.516212 (0.660456) accuracy= 0.000000 (0.002627) accuracy_back= 1.000000 (0.986008) mean_time= 0.332782s

The loss is relatively big since at the end of the training process, the segmentation loss should be around 0.01 ~ 0.03. Just train the model for more episodes.


is it updated flutter 2.8.0 ?

Any update on flutter 2.8.0?

Yes, major update on Camera

Our SDK use native API to access the camera, I think It is not affected.

Okay. But did you look up 2.8 updates?

Okay. But did you look up 2.8 updates?

Im using 2.8, and 4.0.7 seems to work just fine


arch/xtensa/src/esp32/esp32_cpustart.c: nxstyle fixes

The remaining errors:
Mixed case identifier found

I didn't fix them because they were on ROM symbols,
which are not supposed to obey NuttX's coding style.

my suggestion is to hardcode whitelists in nxstyle.

There already are some hard coded mixed case identifiers, for system calls for instance.
But those are, standardized and used throughout the code base.
In this case, the identifiers are only Xtensa related.

@Ouss4
right. do you have any alternative suggestions?

@yamt
Perhaps have a look at the ld file (boards/xtensa/esp32/esp32-core/scripts/esp32_rom.ld) and change the symbol names to lowercase.

@johannes-nivus i believe that file is from esp-idf. not "ours".

@yamt I like the idea of white lists.  @Ouss4 but I agree that a supset of all while listed variables for all architectures is not practical or maintainable.  How about (1) an internal whilelist inside of nxstyle that will accept common mixed case sysmbols, and (1) and optional command line argument to refer to a file of additional whitlisted names?

I like the idea of white lists.

What about a comment section that can signal allowed variable names that would otherwise violate coding standard?
That would keep it self containing.

There is nothing we can do to the nxstyle complaints at this point.
I suggest we go ahead and merge this. The changes are only cosmtics it doesn't need to be rebased to kick the CI off.
Let's move the white list discussion to a separate "issue".

I suggest we go ahead and merge this.

I agree.  The only downside is that the nxstyle checks have blocked the other, more important tests.  I believe that this is a PR to fix this, but is has not been merged.
I will go ahead and do the merge so that this does not linger as an Open PR.

Please see #569 .


Socket.prototype._handle is deprecated warning is caused by MDNS.ts

Describe the bug
Inside MDNS.ts, it's using _handle in a few places. In particular:
        if (platform === 'linux' && disableLinuxMulticastAllOption) {
          socketUtils.disableSocketMulticastAll(
            (unicastSocket as any)._handle.fd,
          );
        }

          if (platform === 'linux' && disableLinuxMulticastAllOption) {
            socketUtils.disableSocketMulticastAll((socket as any)._handle.fd);
          }

The usage of _handle in the Socket prototype causes warnings.
It looks like this in PK CLI:
(node:500471) [DEP0112] DeprecationWarning: Socket.prototype._handle is deprecated
(Use `node --trace-deprecation ...` to show where the warning was created)

Right after starting the agent.
Find an alternative way to do the above, or we need some way of disabling the warning? This is some sort of internal property right?
Apparently Node allows disabling warnings on startup.
To Reproduce

Run Polykey-CLI polykey agent start
Observe warning in the terminal

Expected behavior
No warnings should occur.
Additional context

https://nodejs.org/api/deprecations.html#DEP0112

Notify maintainers
@amydevs

One way to get around this warning is potentially doing all the work in the C++ level? One way to do this is pass the reference to the JS socket object to the native code side, and get the native code to do the necessary referencing of the underlying FD.
Also can we start changing all our native code to Rust to align with js-quic and js-exec? I know js-db is still pending, but it's usually better.
Look at node native code docs as to how to directly interact with socket objects.

If we are only using the net.Socket._handle to extract the fd of the socket, can't we just create the socket with a predetermined fd and reference that wherever we need to?
The new net.Socket() constructor has an option to specify the fd. We can use the C++ socketUtils to find the closest unoccupied file descriptor, and use that while construction, and keep track of that in a Map or a Record. We can then later reference that when we need to call disableSocketMulticastAll.
https://nodejs.org/api/net.html#new-netsocketoptions

I'm not really sure what that means. You'd have to try it. But my understanding is that we are creating sockets for this purpose and so using the rust native code is better anyway to align all of our native libraries.

After looking over the source code a bit more, my solution feels more like a temporary patch than a permanent solution. As I couldn't find much information about this online, I asked (ChatGPT)[https://chatgpt.com/share/e/67344a0d-7da0-800d-9921-29d7a2b65216], and it agreed with you that creating a native binding which manages the entire socket lifetime seems to be the best solution, as no API currently exists which allows us to do the fine-grained multicast manipulation or get the fd without relying on _handle.

@aryanjassal i've tried to implement the solution you had suggested originally when working on it in the https://github.com/MatrixAI/js-mdns/tree/feature-remove-warning branch. However, I was not able to successfully get the node socket to correctly attach to the fd of the existing socket. There should be a test written in that repo that demonstrates this behaviour.

If I recall, the socket modification was to avoid some extra logic in the js-mdns library to handle it. We may just have to do that in the end to resolve this. @amy I don't recall the exact details of this. Can you give a summary of why the socket was modified leading to this warning and what it was trying to avoid in the code. So basically how need to fix this.

It's cause disableSocketMulticastAll needs to access to the fd number which is hidden/encapsulated within the socket APIs.
Therefore I believe it's better to go down one layer into the C++ and do it via rust plugin like we do with other native code instead.
Plus doing this would be required to create a proper native MDNS stack that works well with the OS stack anyway (right now our MDNS stack doesn't co-operate with the OS native stack).


ENH: add a message to reset the user position on a motor

This came up at JPLS at NSLS-II.  Due to their geometry and the fact
that their sample is fluid and evaporates, they need to frequently
reset the user position.
attn @prjemian

This is currently registered onto the RE in the JPLS profile, I wanted to hoist this up here to make sure it did not get lost.  I'm on board with holding off merging this for a while.
The innards of set_current_position need to be atomic from the POV of the RunEngine so I don't think we can make it purely a plan.
Maybe we need to add a module-level registry of messages (currently the registry lives on the RE instance so if you want to be a third-party package that sources messages you need to get a hold of the RE that the user is using)?  Maybe entry points?

This is a three line plan in apstools (sending link later).  These steps
Put motor record into SET
write new position to VAL
put into USE
The signals are already past of EpicsMotor
What additional complexity is needed?
On Tue, Feb 18, 2020, 5:46 PM Thomas A Caswell<EMAIL_ADDRESS>wrote:

This is currently registered onto the RE in the JPLS profile, I wanted to
hoist this up here to make sure it did not get lost. I'm on board with
holding off merging this for a while.
The innards of set_current_position need to be atomic from the POV of the
RunEngine so I don't think we can make it purely a plan.
Maybe we need to add a module-level registry of messages (currently the
registry lives on the RE instance so if you want to be a third-party
package that sources messages you need to get a hold of the RE that the
user is using)? Maybe entry points?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/bluesky/bluesky/pull/1301?email_source=notifications&email_token=AARMUMAIREL2XJXRNR3IZODRDRXOXA5CNFSM4KXOKIVKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEMFZJ2Y#issuecomment-587961579,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AARMUMD7R7YGK6MGUWI6NJ3RDRXOXANCNFSM4KXOKIVA
.

What happens if the plan gets interrupted after it is put in SET mode but before it is put back into USE mode?

@prjemian: If interrupted mid-stream, human intervention is required. Automatic recover is not possible, too likely to result in undefined / unintended behavior. That would just be an unavoidable risk of using this Msg type in a plan.

The point of adding this Msg type is to make the possibility of being left in a super weird state as low as possible.

this is covered here now  https://github.com/bluesky/bluesky/pull/1778 , closing


ER: Hardened Docker Build.

This PR has a number of changes:

Config is now a Python dataclass
Gunicorn is now installed with api-requirements.  Flask executes through Gunicorn in the container.
Mutli-staged docker build.  Runtime environment has limited packages and no build packages (reduced attack surface)
Code is namespaced under db_assessment module.
Removed unused imports and sorted imports
Spelling corrections

/gcbrun

/gcbrun

/gcbrun

After the latest round of edits, I have been able to execute this locally as well as submit a manual Cloud Run build.  Removing the draft status on the PR.

/gcbrun

/gcbrun

LGTM


Incorrect value was updated

There was DI=SN set to the "message was routed through the HRDP". This is my first time i've noticed incorrect Microsoft Article. I had submit a request to the support to change it to DI=SO as my research says it should be DI=SO

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: b58f8e2c-428a-4d8e-e8b7-a0342a0038db
Version Independent ID: 6e7fab1b-1e24-9e36-176d-c3fb14f233a7
Content: Message trace in the Security & Compliance Center - Office 365
Content Source: microsoft-365/security/office-365-security/message-trace-scc.md
Service: o365-seccomp
GitHub Login: @chrisda
Microsoft Alias: chrisda

@raaghavvv, are you saying the topic as currently written is correct or incorrect?

It’s correct now, it’s updated on Friday. Prior to that it was incorrectly updated, I had to open a ticket with Microsoft to correct it.

@raaghavvv - Thank you for submitting feedback.
From our understanding, the issue has been resolved based on this merged commit https://github.com/MicrosoftDocs/microsoft-365-docs/commit/0341d95.
Thank you for your contribution to make the docs better! Much appreciated!


nixos: now beta

Motivation for this change
I'll tag it once it gets merged.
cc @worldofpeace
Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[ ] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[ ] Fits CONTRIBUTING.md.

cc @worldofpeace

@jonringer Thanks, this looks good.


Issue with CLI :

I installed the lightning CLI on VS Code with npm install -g @lightningjs/cli i
To check if the installation was successful I ran **lng -V**, i got the following is the error:
lng : The term 'lng' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included,
verify that the path is correct and try again.
At line:1 char:1

lng create

  + CategoryInfo          : ObjectNotFound: (lng:String) [], CommandNotFoundException
  + FullyQualifiedErrorId : CommandNotFoundException

Can you help as it is not documented anywhere how to fix this

@frank-weindel Thanks a lot. npx lng -v and npx lng generate worked. Thanks @uguraslan as well.


Set the jvm arguments of the gradle compiler deamon

I am trying to get the Groovy project compiling using Gradle 3.2.1 under jigsaw 155. This time I got blocked by the Gradle Compiler Daemon trying to handle the build scripts. I know how to fix the problem (by adding exports as jvm argument), but I have not found a way to set the jvm arguments of the Gradle Compiler Daemon worker. Here is at least documentation missing. A possible workaround would be to force the build source compilation be part of the same JVM as gradle itself or anything that let´s me set options via GRADLE_OPTS. But I did not find any information on if that is possible either.

@ghale IIRC there was recent work in this area, correct? Do you have any information for @blackdrag here?

You should be able to do this by setting the fork options on the compile task.  For instance:
compileJava {
  options.forkOptions.jvmArgs += "-foo"
}

This is documented through the DSL reference (https://docs.gradle.org/current/dsl/org.gradle.api.tasks.compile.JavaCompile.html) although it would probably require some digging.

Right, thanks Gary. For some reason I had it in my head that that didn't affect the compiler daemons.
Closing as answered. Please reopen @blackdrag if anything is amiss.

I don´t think I have the access rights to reopen the issue. Because I already had this before I filed this issue:
compileJava {
    dependsOn ensureGrammars
   //    options.fork(memoryMaximumSize: javacMain_mx)
   options.forkOptions.jvmArgs << "---fooo"
}

What I had expected to happen was the build script compiler daemon to fail to start because of the invalid JVM option. That did not happen, nor did the option show up in the log:

08:34:31.953 [DEBUG] [org.gradle.process.internal.worker.child.WorkerProcessClassPathProvider] Using worker process classpath: [/home/blackdrag/.gradle/caches/3.2.1/workerMain/gradle-worker.jar]
08:34:31.967 [INFO] [org.gradle.process.internal.DefaultExecHandle] Starting process 'Gradle Compiler Daemon 1'. Working directory: /home/blackdrag/coding/groovy/commit/buildSrc Command: /opt/jdk9-jigsaw-ea-155/bin/java @/tmp/gradle-worker-classpath1103722571087631810txt -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant worker.org.gradle.process.internal.worker.GradleWorkerMain 'Gradle Compiler Daemon 1'
08:34:31.967 [DEBUG] [org.gradle.process.internal.DefaultExecHandle] Environment for process 'Gradle Compiler Daemon 1': {PATH=/opt/jdk8/bin:/home/blackdrag/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games, LC_MEASUREMENT=de_DE.UTF-8, XAUTHORITY=/home/blackdrag/.Xauthority, LC_TELEPHONE=de_DE.UTF-8, XMODIFIERS=@im=ibus, GRADLE_OPTS=--add-opens java.base/java.lang=ALL-UNNAMED --add-opens java.base/java.util=ALL-UNNAMED, MANDATORY_PATH=/usr/share/gconf/mate.mandatory.path, GDMSESSION=mate, XDG_DATA_DIRS=/usr/share/mate:/usr/local/share/:/usr/share/, LC_TIME=de_DE.UTF-8, TEXTDOMAINDIR=/usr/share/locale/, GTK_IM_MODULE=ibus, DBUS_SESSION_BUS_ADDRESS=unix:abstract=/tmp/dbus-ygCln4ToB4,guid=485233b321c5a4ebe00eb14958b863ce, DEFAULTS_PATH=/usr/share/gconf/mate.default.path, XDG_CURRENT_DESKTOP=MATE, SSH_AGENT_PID=1919, COLORTERM=mate-terminal, QT4_IM_MODULE=xim, LC_PAPER=de_DE.UTF-8, SESSION_MANAGER=local/Gr8:@/tmp/.ICE-unix/1841,unix/Gr8:/tmp/.ICE-unix/1841, LOGNAME=blackdrag, PWD=/home/blackdrag/coding/groovy/commit, LANGUAGE=en_US, WINDOWID=48234533, SHELL=/bin/bash, LESSOPEN=| /usr/bin/lesspipe %s, LC_ADDRESS=de_DE.UTF-8, OLDPWD=/home/blackdrag/coding/groovy/commit, GTK_MODULES=gail:atk-bridge:overlay-scrollbar:canberra-gtk-module, CLUTTER_IM_MODULE=xim, MATE_DESKTOP_SESSION_ID=this-is-deprecated, XDG_SESSION_PATH=/org/freedesktop/DisplayManager/Session0, TEXTDOMAIN=im-config, LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:.tar=01;31:.tgz=01;31:.arj=01;31:.taz=01;31:.lzh=01;31:.lzma=01;31:.tlz=01;31:.txz=01;31:.zip=01;31:.z=01;31:.Z=01;31:.dz=01;31:.gz=01;31:.lz=01;31:.xz=01;31:.bz2=01;31:.bz=01;31:.tbz=01;31:.tbz2=01;31:.tz=01;31:.deb=01;31:.rpm=01;31:.jar=01;31:.war=01;31:.ear=01;31:.sar=01;31:.rar=01;31:.ace=01;31:.zoo=01;31:.cpio=01;31:.7z=01;31:.rz=01;31:.jpg=01;35:.jpeg=01;35:.gif=01;35:.bmp=01;35:.pbm=01;35:.pgm=01;35:.ppm=01;35:.tga=01;35:.xbm=01;35:.xpm=01;35:.tif=01;35:.tiff=01;35:.png=01;35:.svg=01;35:.svgz=01;35:.mng=01;35:.pcx=01;35:.mov=01;35:.mpg=01;35:.mpeg=01;35:.m2v=01;35:.mkv=01;35:.webm=01;35:.ogm=01;35:.mp4=01;35:.m4v=01;35:.mp4v=01;35:.vob=01;35:.qt=01;35:.nuv=01;35:.wmv=01;35:.asf=01;35:.rm=01;35:.rmvb=01;35:.flc=01;35:.avi=01;35:.fli=01;35:.flv=01;35:.gl=01;35:.dl=01;35:.xcf=01;35:.xwd=01;35:.yuv=01;35:.cgm=01;35:.emf=01;35:.axv=01;35:.anx=01;35:.ogv=01;35:.ogx=01;35:.aac=00;36:.au=00;36:.flac=00;36:.mid=00;36:.midi=00;36:.mka=00;36:.mp3=00;36:.mpc=00;36:.ogg=00;36:.ra=00;36:.wav=00;36:.axa=00;36:.oga=00;36:.spx=00;36:.xspf=00;36:, SHLVL=1, LC_IDENTIFICATION=de_DE.UTF-8, LC_MONETARY=de_DE.UTF-8, LESSCLOSE=/usr/bin/lesspipe %s %s, COMPIZ_CONFIG_PROFILE=mate, QT_IM_MODULE=ibus, JAVA_HOME=/opt/jdk8, TERM=xterm, XDG_CONFIG_DIRS=/etc/xdg/xdg-mate:/etc/xdg, GNOME_KEYRING_CONTROL=/run/user/1000/keyring-giXpos, XDG_SEAT_PATH=/org/freedesktop/DisplayManager/Seat0, LANG=en_US.UTF-8, XDG_SESSION_ID=c2, QT_STYLE_OVERRIDE=gtk, DISPLAY=:0.0, LC_NAME=de_DE.UTF-8, GDM_LANG=en_US, XDG_GREETER_DATA_DIR=/var/lib/lightdm-data/blackdrag, GPG_AGENT_INFO=/run/user/1000/keyring-giXpos/gpg:0:1, DESKTOP_SESSION=mate, USER=blackdrag, QT_ACCESSIBILITY=1, LC_NUMERIC=de_DE.UTF-8, SSH_AUTH_SOCK=/run/user/1000/keyring-giXpos/ssh, XDG_SEAT=seat0, QT_QPA_PLATFORMTHEME=appmenu-qt5, XDG_VTNR=7, XDG_RUNTIME_DIR=/run/user/1000, HOME=/home/blackdrag, GNOME_KEYRING_PID=1839}

If this is supposed to work for Gradle 3.2.1, then it has a bug. I just checked against 3.4 and there we have:

08:42:03.902 [DEBUG] [org.gradle.process.internal.worker.child.WorkerProcessClassPathProvider] Using worker process classpath: [/home/blackdrag/.gradle/caches/3.4/workerMain/gradle-worker.jar]
08:42:03.921 [INFO] [org.gradle.process.internal.DefaultExecHandle] Starting process 'Gradle Worker Daemon 1'. Working directory: /home/blackdrag/coding/groovy/commit/buildSrc Command: /opt/jdk9-jigsaw-ea-155/bin/java @/tmp/gradle-worker-classpath7652957163780460905txt -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant worker.org.gradle.process.internal.worker.GradleWorkerMain 'Gradle Worker Daemon 1'
08:42:03.922 [DEBUG] [org.gradle.process.internal.DefaultExecHandle] Environment for process 'Gradle Worker Daemon 1': {PATH=/opt/jdk8/bin:/home/blackdrag/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games, LC_MEASUREMENT=de_DE.UTF-8, XAUTHORITY=/home/blackdrag/.Xauthority, LC_TELEPHONE=de_DE.UTF-8, XMODIFIERS=@im=ibus, GRADLE_OPTS=--add-opens java.base/java.lang=ALL-UNNAMED --add-opens java.base/java.util=ALL-UNNAMED, MANDATORY_PATH=/usr/share/gconf/mate.mandatory.path, GDMSESSION=mate, XDG_DATA_DIRS=/usr/share/mate:/usr/local/share/:/usr/share/, LC_TIME=de_DE.UTF-8, TEXTDOMAINDIR=/usr/share/locale/, GTK_IM_MODULE=ibus, DBUS_SESSION_BUS_ADDRESS=unix:abstract=/tmp/dbus-ygCln4ToB4,guid=485233b321c5a4ebe00eb14958b863ce, DEFAULTS_PATH=/usr/share/gconf/mate.default.path, XDG_CURRENT_DESKTOP=MATE, SSH_AGENT_PID=1919, COLORTERM=mate-terminal, QT4_IM_MODULE=xim, LC_PAPER=de_DE.UTF-8, SESSION_MANAGER=local/Gr8:@/tmp/.ICE-unix/1841,unix/Gr8:/tmp/.ICE-unix/1841, LOGNAME=blackdrag, PWD=/home/blackdrag/coding/groovy/commit, LANGUAGE=en_US, WINDOWID=48234533, SHELL=/bin/bash, LESSOPEN=| /usr/bin/lesspipe %s, LC_ADDRESS=de_DE.UTF-8, OLDPWD=/home/blackdrag/coding/groovy/commit, GTK_MODULES=gail:atk-bridge:overlay-scrollbar:canberra-gtk-module, CLUTTER_IM_MODULE=xim, MATE_DESKTOP_SESSION_ID=this-is-deprecated, XDG_SESSION_PATH=/org/freedesktop/DisplayManager/Session0, TEXTDOMAIN=im-config, LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:.tar=01;31:.tgz=01;31:.arj=01;31:.taz=01;31:.lzh=01;31:.lzma=01;31:.tlz=01;31:.txz=01;31:.zip=01;31:.z=01;31:.Z=01;31:.dz=01;31:.gz=01;31:.lz=01;31:.xz=01;31:.bz2=01;31:.bz=01;31:.tbz=01;31:.tbz2=01;31:.tz=01;31:.deb=01;31:.rpm=01;31:.jar=01;31:.war=01;31:.ear=01;31:.sar=01;31:.rar=01;31:.ace=01;31:.zoo=01;31:.cpio=01;31:.7z=01;31:.rz=01;31:.jpg=01;35:.jpeg=01;35:.gif=01;35:.bmp=01;35:.pbm=01;35:.pgm=01;35:.ppm=01;35:.tga=01;35:.xbm=01;35:.xpm=01;35:.tif=01;35:.tiff=01;35:.png=01;35:.svg=01;35:.svgz=01;35:.mng=01;35:.pcx=01;35:.mov=01;35:.mpg=01;35:.mpeg=01;35:.m2v=01;35:.mkv=01;35:.webm=01;35:.ogm=01;35:.mp4=01;35:.m4v=01;35:.mp4v=01;35:.vob=01;35:.qt=01;35:.nuv=01;35:.wmv=01;35:.asf=01;35:.rm=01;35:.rmvb=01;35:.flc=01;35:.avi=01;35:.fli=01;35:.flv=01;35:.gl=01;35:.dl=01;35:.xcf=01;35:.xwd=01;35:.yuv=01;35:.cgm=01;35:.emf=01;35:.axv=01;35:.anx=01;35:.ogv=01;35:.ogx=01;35:.aac=00;36:.au=00;36:.flac=00;36:.mid=00;36:.midi=00;36:.mka=00;36:.mp3=00;36:.mpc=00;36:.ogg=00;36:.ra=00;36:.wav=00;36:.axa=00;36:.oga=00;36:.spx=00;36:.xspf=00;36:, SHLVL=1, LC_IDENTIFICATION=de_DE.UTF-8, LC_MONETARY=de_DE.UTF-8, LESSCLOSE=/usr/bin/lesspipe %s %s, COMPIZ_CONFIG_PROFILE=mate, QT_IM_MODULE=ibus, JAVA_HOME=/opt/jdk8, TERM=xterm, XDG_CONFIG_DIRS=/etc/xdg/xdg-mate:/etc/xdg, GNOME_KEYRING_CONTROL=/run/user/1000/keyring-giXpos, XDG_SEAT_PATH=/org/freedesktop/DisplayManager/Seat0, LANG=en_US.UTF-8, XDG_SESSION_ID=c2, QT_STYLE_OVERRIDE=gtk, DISPLAY=:0.0, LC_NAME=de_DE.UTF-8, GDM_LANG=en_US, XDG_GREETER_DATA_DIR=/var/lib/lightdm-data/blackdrag, GPG_AGENT_INFO=/run/user/1000/keyring-giXpos/gpg:0:1, DESKTOP_SESSION=mate, USER=blackdrag, QT_ACCESSIBILITY=1, LC_NUMERIC=de_DE.UTF-8, SSH_AUTH_SOCK=/run/user/1000/keyring-giXpos/ssh, XDG_SEAT=seat0, QT_QPA_PLATFORMTHEME=appmenu-qt5, XDG_VTNR=7, XDG_RUNTIME_DIR=/run/user/1000, HOME=/home/blackdrag, GNOME_KEYRING_PID=1839}

Which does not look all that different to me and still fails to include my JVM option. It does not even include GRADLE_OPTS. I know the option works for forked javac´s, but the build source is different. And if this is supposed to be the right way, to do it, then consider this a bug report, because it does not

one more thing. On github I can reopen an issue only if I closed it. Since @eriwen closed this issue I cannot reopen it

@blackdrag I assume this is for groovy-core?  I added exactly the configuration you have above to groovy-core/build.gradle and I got a failure to start the compiler daemon.  This happened for the version of gradle that was in the wrapper (2.3), 3.2.1 and 3.4.  The process that was started had the added option:

10:37:19.096 [INFO] [org.gradle.process.internal.DefaultExecHandle] Starting process 'Gradle Compiler Daemon 1'. Working directory: /Users/ghale/repos/groovy-core Command: /Library/Java/JavaVirtualMachines/jdk1.8.0_65.jdk/Contents/Home/bin/java ---fooo -Xmx384m -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant -cp /Users/ghale/.gradle/wrapper/dists/gradle-2.3-bin/7a1ngj6cc7aafikc3uq95v6rn0/gradle-2.3/lib/gradle-base-services-2.3.jar:/Users/ghale/.gradle/wrapper/dists/gradle-2.3-bin/7a1ngj6cc7aafikc3uq95v6rn0/gradle-2.3/lib/gradle-core-2.3.jar:/Users/ghale/.gradle/wrapper/dists/gradle-2.3-bin/7a1ngj6cc7aafikc3uq95v6rn0/gradle-2.3/lib/gradle-cli-2.3.jar:/Users/ghale/.gradle/wrapper/dists/gradle-2.3-bin/7a1ngj6cc7aafikc3uq95v6rn0/gradle-2.3/lib/gradle-native-2.3.jar:/Users/ghale/.gradle/wrapper/dists/gradle-2.3-bin/7a1ngj6cc7aafikc3uq95v6rn0/gradle-2.3/lib/gradle-messaging-2.3.jar:/Users/ghale/.gradle/wrapper/dists/gradle-2.3-bin/7a1ngj6cc7aafikc3uq95v6rn0/gradle-2.3/lib/slf4j-api-1.7.7.jar:/Users/ghale/.gradle/wrapper/dists/gradle-2.3-bin/7a1ngj6cc7aafikc3uq95v6rn0/gradle-2.3/lib/logback-classic-1.0.13.jar:/Users/ghale/.gradle/wrapper/dists/gradle-2.3-bin/7a1ngj6cc7aafikc3uq95v6rn0/gradle-2.3/lib/logback-core-1.0.13.jar:/Users/ghale/.gradle/wrapper/dists/gradle-2.3-bin/7a1ngj6cc7aafikc3uq95v6rn0/gradle-2.3/lib/jul-to-slf4j-1.7.7.jar:/Users/ghale/.gradle/wrapper/dists/gradle-2.3-bin/7a1ngj6cc7aafikc3uq95v6rn0/gradle-2.3/lib/guava-jdk5-17.0.jar org.gradle.process.internal.launcher.GradleWorkerMain 'Gradle Compiler Daemon 1'

I'm guessing something else is going on here (possibly something overwriting jvmArgs later in build script?).  Is your not-working configuration on a branch or somewhere that I could take a look at it?

This is for groovy-core, yes, and I will try soon. But could you explain "in process" a bit more? Because it is not the process I start with the gradle command, that I can influence using GRADLE_OPTS. Why is gradle creating separate process for the builder daemon, but lets you set options for it only in fork mode?

ok, after trying out for a few minutes... Is it possible this behaves different on jdk8 and jdk9? Because my "---fooo" seems not to appear using:
Java(TM) SE Runtime Environment (build 9-ea+155-jigsaw-nightly-h6059-20170208)
Java HotSpot(TM) 64-Bit Server VM (build 9-ea+155-jigsaw-nightly-h6059-20170208, mixed mode)

an I did just see I still have artefacts from several tries before...  So I also have
org.gradle.jvmargs=-ea -Xmx1G --add-opens java.base/java.lang=ALL-UNNAMED --add-opens java.base/java.util=ALL-UNNAMED
#org.gradle.jvmargs=-Xmx1G -XX:MaxPermSize=384m -XX:+CMSClassUnloadingEnabled -XX:+CMSPermGenSweepingEnabled

in gradle.settings and also org.gradle.daemon=false. But as you can see setting the daemon to false does not prevent the creation of the builder daemon. Which leads me back to not understanding the handling of this build script daemon. I mean why is a fork option on compileJava controlling (yet to be verified) the buildscript daemon? Why can`t I disabled the usage of daemons with org.gradle.daemon=false anymore? why does the build script daemon ignore  org.gradle.jvmargs? Why does the daemon not take GRADLE_OPTS?
You see I am hopelessly lost and I am probably circling around a bit because I have no clue about how things are working anymore and it would help understanding how things are supposed to be working at least. At this point I can´t even tell if the forked JVMs for compileGroovy or compileJava have anything to do with the one for the build script. My assumption would have been that I can set these separately, but your suggestion is telling me other things.  I would have thought for example that we could have the build script compilation even without a java or groovy compiling build. How does it then make sense to use compileJava fork options to control this?
Really, not ranting here, just trying to understand the workings of this build script daemon.

I got the same issues here with the latest Java(TM) SE Runtime Environment (build 9-ea+158) preview build.
There is a epic #890 that should add support for JDK 9 also, unfortunately there where no actual updates since January. I would like to help out in testing more in this area, but did not jet receive any response jet.
At last, I  saw that you still use the -XX:MaxPermSize=384m parameter. I just wanted to to point out, that this option has been removed since JDK 8, due the fact that Perm memory is no longer available from that JDK on. A similar option would be -XX:MaxMetaspaceSize=384m

@reinhapa I actually used only
org.gradle.jvmargs=-ea -Xmx1G --add-opens java.base/java.lang=ALL-UNNAMED --add-opens java.base/java.util=ALL-UNNAMED

I've been working with JDK9 since the early builds.  The restriction on setAccessible was a big step backward for Gradle.  While it's possible to add --add-opens to the environment generally, it does open it up for other software beside Gradle.  On the other hand, trying to change every project build.gradle to have the right packages opened is very invasive.    The solution that I found worked best in our environment was to introduce a shell script to invoke Gradle.  In the shell script, when running with JDK9 I export _JAVA_OPTIONS with all of the --add-opens arguments needed for all of the projects in our build.  It's also a good place to add -Dsun.reflect.debugModuleAccessChecks=true to track down additional packages that need to be added.
This isolates changes to this shell and the opened packages only to gradle and stuff executed under gradle.
I was trying to figure out if there is anything else that is needed (the environment is pretty complex).
We do have
tasks.withType(JavaCompile) {
options.fork = true
options.forkOptions.executable = 'javac'  // we have this in one build tree but not another
options.encoding = 'UTF-8'
}
We got rid of all JDK9 arguments from the compilerArgs and jvmArgs - much too complicated.
Meanwhile, we would really like to see Gradle and Groovy fixed.  I know that this approach to open packages is likely hiding a few more changes that are needed in the product code.

@sdfelts Goal of this is to be able to run a build with JDK9. Fixing general JDK9 problems of Groovy is not the goal here. For those it is probably better to run gradle with a JDK8 and run all the forked compilations and tests with a different JDK:
test {
   executable = "/path/to/jdk/bin/java"
}

and for compilation you already provided a way to use a different JDK. The Groovy problems with JDK9 I have seen so far (just normal classpath based projects without additional modules) seem not to have a problem with recent Groovy. I am very sure that is only because of the reluctant testing though. But for Groovy errors in Gradle...most of those are because here Groovy was used to bypass restrictions. There is no fix for these on the Groovy side short of rewriting java module descriptors on startup. And that is because of the way the module system is designed and its additional restrictions it imposes on everyone. And actually non of these problems are actually specific to Groovy. For most of these solutions you would probably have used reflection and it would have failed by now as well.

@blackdrag Any new findings or even solutions regarding the "compiler daemon" issue? I'm stuck at a similar build-show-stopper here: https://travis-ci.org/junit-team/junit5/builds/219179434#L1029

@sormuras try using a very recent jigsaw build and then

JAVA_OPTS='-XX:+IgnoreUnrecognizedVMOptions --permit-illegal-access'

plus in the gradle.properties:

org.gradle.jvmargs=-XX:+IgnoreUnrecognizedVMOptions --permit-illegal-access --show-version

I had not yet the time to actually try this out though. If you did try this, it would be very nice if you could report if it was of any help

And leave the daemon "alive and kicking" enabled?

as I mentioned in my post above on March 7, I use _JAVA_OPTIONS to get JDK9 command line arguments from the command line to all embedded processes.  We call gradle from the command line, ant, shell, etc. There are no problems running gradle or the related daemons (no need to turn them off using --no-daemon).
When b163 arrived, I put in a switch so that I can either run with the single --permit-illegal-access or the twenty five or so--add-opens.  The --permit-illegal-access has the disadvantage that it prints out warnings even for places in the code that have fall-back code when the setAccessible fails.  I've been building up a list of known cases that I can ignore.  It's a great way to run the first time to get all of the problem packages in one pass instead of fixing one and re-running to the next one (annoying if you have hundreds of modules).

This is my gradle shell.  It adds JDK9 options for running gradle to the other options needed for my project code.  That allows me to isolate as much as possible the gradle/groovy dependencies.
We happen to have the Java version in an environment variable but you could always get it from java -version each time.
Note that the java.lang.ClassLoader.getPackages() problem mentioned in the link above is solved by using the --add-opens=java.base/java.lang=ALL-UNNAMED option or --permit-illegal-access.
The --add-opens that you need will depend on the classes and methods used in your gradle code.  We have a lot of code and a lot of dependencies.
if [ "uname" = "Windows_NT" ]; then
GRADLE=gradle.bat
else
GRADLE=gradle
fi
case "$JAVAVER" in
1900*)
if [ x"$_JAVA_OPTIONS" = x--permit-illegal-access ]
then
_JAVA_OPTIONS=--permit-illegal-access
else
_JAVA_OPTIONS="$_JAVA_OPTIONS --add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.lang.invoke=ALL-UNNAMED --add-opens=java.base/java.lang.reflect=ALL-UNNAMED --add-opens=java.base/java.net=ALL-UNNAMED --add-opens=java.base/java.text=ALL-UNNAMED --add-opens=java.base/java.util=ALL-UNNAMED --add-opens=java.base/java.util.concurrent=ALL-UNNAMED --add-opens=java.base/java.util.regex=ALL-UNNAMED --add-opens=java.base/java.util.jar=ALL-UNNAMED --add-opens=java.base/java.util.zip=ALL-UNNAMED --add-opens=java.base/javax.net=ALL-UNNAMED --add-opens=java.base/sun.net.www=ALL-UNNAMED --add-opens=java.base/sun.net.www.protocol.file=ALL-UNNAMED --add-opens=java.base/sun.nio.fs=ALL-UNNAMED --add-opens=java.xml/javax.xml.transform=ALL-UNNAMED --add-opens=java.xml/javax.xml.transform.stream=ALL-UNNAMED --add-opens=java.xml/com.sun.org.apache.xalan.internal.xsltc=ALL-UNNAMED --add-opens=java.xml/com.sun.org.apache.xalan.internal.xsltc.compiler=ALL-UNNAMED --add-opens=java.xml/com.sun.org.apache.xalan.internal.xsltc.trax=ALL-UNNAMED --add-opens=java.base/java.security=ALL-UNNAMED"
fi
#_JAVA_OPTIONS="$_JAVA_OPTIONS -Dsun.reflect.debugModuleAccessChecks=true"
export _JAVA_OPTIONS
;;
esac
"$GRADLE_HOME/bin/$GRADLE" "$@"

@sdfelts Thanks for the comments and hints. If I read your shell script and understand its usage correctly, it is used to initially start gradle with the vm options needed/wanted.
But what about the compile daemons @ghale talks about? What about vms that are spawend internally by Gradle? Subprojects, buildSrc folder? I can't get said vm options injected into those vms -- no matter if let the daemon live or not.
See, we have a "green" build with "--no-daemon": https://travis-ci.org/junit-team/junit5/builds/219396869
When I re-activate the junit-platform-gradle-plugin: https://travis-ci.org/junit-team/junit5/builds/219194962
The "opens" are no longer active, they're not passed down to the spawned "compileGroovy" task.

@sormuras setting the fork options for groovy compile does not work?

seems using _JAVA_OPTIONS works. Please not the leading underscore

I have been using _JAVA_OPTIONS to get gradle working for over a year.
Gradle seems to pass this environment to any daemons.
The only change that we made is  to fork java (see the March 7 comment).

@sormuras setting the fork options for groovy compile does not work?

It did not work. But after your " seems using _JAVA_OPTIONS works " I didn't investigate any further that path.

I have been using _JAVA_OPTIONS to get gradle working for over a year.

Thanks @sdfelts ! I totally forgot about that JDK built-in (undocumented?) feature -- even after you mentioned it here many times. Needed time to digest it. Now, the JUnit 5 build on Java 9 is "green" again: https://travis-ci.org/junit-team/junit5/builds/219592861

While _JAVA_OPTIONS is unsupported, there is now a supported equivalent.
It's not documented yet.  The functionality is documented at http://ccc.us.oracle.com/8170832 but the official new name is JDK_JAVA_OPTIONS.  It's available in b157.  It prepends to the command line instead of appends and it supports @filename.   I haven't tried it yet but I will be switching over soon (limiting us to run on b157 or later).

Found the "external" link to the JDK_JAVA_OPTIONS issue: https://bugs.openjdk.java.net/browse/JDK-8173712
Incorporated it into our build already: https://travis-ci.org/junit-team/junit5/builds/219975618#L313
Thanks, again!

The final JDK 9 build (b181) defaults to having all packages opened.  All of these workarounds are no longer needed.  However, a warning is still printed during the gradle code execution and it should be fixed.

Thank you for verifying, let's continue on that other issue then.

while the jdk9 specific problem here got solved by jdk9 diverging from the originally planed logic - I still cannot set the jvm arguments of the gradle compiler daemon. I have not tested with Gradle 4.1 though.

The compiler daemons are configured via the options on the compile task. If that doesn't work, can you attach a small example that shows the problem?

How to get memory dump with gradle daemon and javac? The next snippet doesn't work.
  tasks.withType(JavaCompile).all {
    options.forkOptions.jvmArgs += '-XX:+HeapDumpOnOutOfMemoryError'
  }

In the case you want to go to the add-open JVM option, here's a command to find which module provide which package ->
java --list-modules | tr @ " " | awk '{ print $1 }' | xargs -n1 java -d
the name of the module will be shown with the @ while the name of the packages without it
(tested with JDK 11)


More init changes - solarflare NIC

What this PR does / why we need it:
Which issue this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close that issue when PR gets merged): fixes #
Special notes for your reviewer:
Release note:

Why are you deleting manager test ?

Also please rebase over master so I can merge :)


build: update ioredis

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

0 of 0   changed or added relevant lines in 0 files are covered.
7 unchanged lines in 2 files lost coverage.
Overall coverage decreased (-1.6%) to 96.503%

Files with Coverage Reduction
New Missed Lines
%

packages/core/src/index.js
2
95.17%

packages/sql/src/index.js
5
95.3%

Totals

Change from base Build<PHONE_NUMBER>:
-1.6%

Covered Lines:
892

Relevant Lines:
910

💛  - Coveralls


Introduce consistent tracing / metrics functionality across client libraries

We want a consistent tracing / metrics story across all Java client libraries, and this should be a similar story (although different implementation) for all client SDKs.
There are a few different things to consider:
1. Handling dependencies
We should not have any dependency on a third-party library (such as OpenCensus) hard-coded into our POMs. We should instead configure our HttpPipeline to discover this and automatically include the necessary http policies automatically. We should never have any import statement that includes OpenCensus or other libraries in our client library code.
Todo:

[ ] Decide whether the presence of a pipeline policy is sufficient for that policy to be automatically enabled, or whether it also requires the user to manually add it to their pipeline as well.
[ ] Create CheckStyle rules to ensure there are no import statements from OpenCensus leaking into any code, other than specific OpenCensus classes that will be suppressed.
[ ] Create OpenCensus http policy
[ ] Create Java SPI to dynamically discover an OpenCensus provider and to include it in the pipeline. This may be made more generic to discover service providers for pipeline policies that are injected before or after the retry policy. Proof of concept implementation exists as a draft PR.

2. Provide a means to enable end-users to provide tracing context
To allow tracing to work, developers must be able to pass in contextual information related to the current span. There are currently three approaches to enabling this that should be explored:
1. Explicit augmentation of service client API to receive a context that end-users may insert the span information into. This requires developers to understand yet-another service client API argument, or it will require an overloaded method variant to not overwhelm users with this.
2. Byte-code manipulation to insert this information into an appropriate location (e.g. ThreadLocal). This removes the need for an additional method argument on all service client APIs, but this does not work as desired in the async case.
3. In the async case, we may be able to make use of the Project Reactor Context functionality to pass in the span details without the need for an additional method parameter.
Todo:

[ ] Update Java specification to clarify how tracing should be supported
[ ] Update existing track two client libraries to follow this spec
[ ] Add any relevant checkstyle rules that can be used to enforce this spec

3. Provide tracing information within each client library service method
In cases where a user does not pass in any tracing information, we should create this tracing information ourselves, and include it in the context. Obviously, this should only happen when a tracing provider is available, and this work should be implemented that tracing provider.
Todo:

[ ] Gather requirements about what is required to build a span internally to our client libraries. What information must be placed in a span, for example?
[ ] Specify another SPI to build spans, abstracted behind new API in azure-core. This azure-core library will short-circuit if no tracing provider is present.
[ ] Update all sync and async client libraries to call this azure-core API.

Mockup of potential implementation

User calls into 'getFoo' method of a service client. This call will include a general-purpose Context object that may or may not have tracing information included.
The 'getFoo' method will immediately defer to a 'Tracing API' in azure-core. This API will check if a tracing service provider exists. If one does not exist, it returns immediately. If one exists, it will forward the Context object on to the tracing service provider.
The tracing service provider will receive the Context object, inspect it for any existing tracing information, and augment it with any additional tracing information provided by the method.

At this point, the Context object will have been updated, and control will return to the 'getFoo' method to perform other work. Once it is complete and the request goes into the HTTP pipeline, again, the Context object may pass through a tracing pipeline policy for a specific tracing library (e.g. OpenCensus), at which point the values will be read out again, and added into the outgoing request in the appropriate manner.

Byte-code manipulation to insert this information into an appropriate location (e.g. ThreadLocal). This removes the need for an additional method argument on all service client APIs, but this does not work as desired in the async case.

The byte-code manipulation requires the app to run with the byte-code agent attach while may not be desirable/discoverable in many cases. I.e. this does not solve a generic context propagation problem.

There are many potential uses for an additional method parameter. If you think of it as a general-purpose parameter, then it can be used as a mechanism allowing a caller to override some behavior on a per-method basis. In Go, there is a generic "context" object that is passed to every method; this context controls (per-method) timeout and also a set of key/value pairs that the customer can set and the pipeline can query. We used this context.WithValue mechanism a lot in AzCopy  to control API version numbers when calling Azure Stack as opposed to public Azure. This same mechanism will be used for distributed tracing spans.
If you look at the old Storage client libraries (in all languages), you'll see that all the methods accept a XxxRequestOptions parameter like BlobRequestOptions. When I redesigned the Storage client libraries I was able to get rid of the need for of the options by introducing the pipeline (and its options) but there are still some options that make sense on a per-method basis. For the Java Storage client library, we added a similar Context class (imported from com.microsoft.rest.v2.Context) that works like Go's (without the timeout support) and created overloads of all methods so this can be passed.
I think this is the best way to deal with spans (and other things) in Java (and other languages). There are other features of Context that are worth understanding (like its code-tree approach and immutability) which makes it better than things like thread local storage or AsyncLocal (in .NET). I'm happy to discuss further.

//cc: @johanste and @bryevdv

Pull request is available for review at #3689


Small Mesh generation Changes

Hi Fritz,
I was wondering if its possible to somehow implement a dropdown which lets you choose between road shoulders or sidewalks which have a little different geometry with a curb but generate the same as shoulders, so that they works with intersections.
The sidewalk extrusion object is very unflexible.
;)

Hello,
we could be adding this to the list of feature requests. Please go ahead to my fork and open an issue.
Looks like you opened this pull request by mistake.
Best Regards,
Fritzs


Edit Project (Post)

Issue
#37
Description
there is a new feature added that is to edit the project (Post)
the user has the option to update anything in the project he/she had posted
there is the button behind the Delete button when it is clicked it will show a popup message with all information about the Project that can be edited and save the changes.
Type of Change:
the file Explore/index.php
and add file in scripts called edit-project.php
How Has This Been Tested?
https://user-images.githubusercontent.com/60848147/114277049-c71f8380-9a29-11eb-8647-bc1a302d7557.mp4

There are some conflicting files in this PR. Please resolve them @Raghad-Khaled

@rupeshmohanty
It has no conflicts now 😀


Macro to safely cast from Scala class to JS trait

Write a macro to do the following with static checking:
trait Foo extends js.Object {
  def foo(x: Int): Int = js.Native
}

class Bar {
  @JSExport
  def foo(x: Int): Int = x * 2
}

val x: Bar = ???

// Macro should check this:
val y: Foo = x.asInstanceOf[Foo]

@sjrd should we have a member asJS that we pimp on everything? Or rather a function in the js namespace?

that's not really a cast is it? That's more a conversion since the data structure will need to be re-created for anything with attributes, which are methods in Scala but raw-attributes in JS?

It is a cast, since Bar has @JSExport on foo. If you have @JSExport on an attribute in Scala, it will be implemented as a property in JS:
class Bar {
  @JSExport
  def a = 1
}

val x = (new Bar).asInstanceOf[js.Dynamic]
x.a // -> 1

What about asInstanceOfs? Those check the prototype chain, and we allow asInstanceOf for Javascript classes

You mean isInstanceOfs. Indeed, they are an issue. But I guess we will only allow the conversion for traits extending js.Any. These do not allow isInstanceOf.

Don't pimp something on everything, ever. This should be a separate function.

Ok. But then we have the issue that we need two type parameters but only one that you specify in general...
We could do:
js.convert(x).to[Foo]
js.asJS(x)[Foo]
js.asJS[Bar, Foo](x)

I don't like any of them... We could protect the pimp with an implicit at the cost of having a sensible error message... :(

Things I have come up with during my first explorations and developments:
Syntax
I propose:
js.use(x).as[Foo]

This allows us to not write the source type if we don't want to. js.use returns a value class, so there is zero overhead to this.
Limitations
I plan to not handle the following things in the first version:
Refinement Types
Given
trait Foo extends js.Object {
  type T
  def get: T = js.native
}

class Bar {
  @JSExport
  def get: Int = 1
}

This will fail:
js.use(new Bar).as[Foo { type T = Int }]

Note that generics (Foo[T]) will be supported.
Multiple Parameter Lists
Given:
trait Foo extends js.Object {
  def do(x: Int, y: Int): Unit = js.native
}

class Bar {
  @JSExport
  def do(x: Int)(y: Int): Unit = js.native
}

This will fail:
js.use(new Bar).as[Foo]

WDYT?

That seems reasonable.


export webp images

Describe the bug
webp images doesn't exporting to pdf or html. But webp images looks good at wiki page =(
Steps To Reproduce
Steps to reproduce the behavior:

Go to 'page with webp image'
Click on 'export as PDF'
See PDF. It will contain error

Expected behavior
A clear and concise description of what you expected to happen.
Screenshots
If applicable, add screenshots to help explain your problem.
Your Configuration (please complete the following information):

Exact BookStack Version (Found in settings): 0.26.2
PHP Version: don't know
Hosting Method (Nginx/Apache/Docker): don't know

P.s.: other image-types exporting fine

Relies upon https://github.com/dompdf/dompdf/issues/1276. Has moved forward recently.


Devfile registry index should contain the starterProjects for each devfile

It would be nice if the index.json for a devfile registry could contain the list of starterProjects in a given devfile:
For odo, when we list devfile components, we want to be able to include the name of the starter projects available in each devfile, e.g:
$ odo catalog list components
Odo Devfile Components:
NAME                 DESCRIPTION                            STARTER PROJECTS                REGISTRY
java-maven           Upstream Maven and OpenJDK 11                                          DefaultDevfileRegistry
java-openliberty     Open Liberty microservice in Java      user-app                        DefaultDevfileRegistry
java-quarkus         Upstream Quarkus with Java+GraalVM     quarkus-ex                      DefaultDevfileRegistry
java-springboot      Spring Boot® using Java                springbootproject               DefaultDevfileRegistry

However, to get the list of projects now would require downloading each devfile and parsing it. If the index.json is updated to include the projects for each devfile, then we would only need to parse the index.json (as-is today).
Sample new index.json:
[
  {
    "name": "java-maven",
    "displayName": "Maven Java",
    "description": "Upstream Maven and OpenJDK 11",
    "tags": [
      "Java",
      "Maven"
    ],
    "projectType": "maven",
    "language": "java",
    "links": {
      "self": "/devfiles/java-maven/devfile.yaml"
    }
  },
  {
    "name": "java-openliberty",
    "displayName": "Open Liberty",
    "description": "Open Liberty microservice in Java",
    "projectType": "docker",
    "language": "java",
    "links": {
      "self": "/devfiles/java-openliberty/devfile.yaml"
    },
    "starterProjects": [
      "user-app"
    ]
  },
  {
    "name": "java-quarkus",
    "displayName": "Quarkus Java",
    "description": "Upstream Quarkus with Java+GraalVM",
    "tags": [
      "Java",
      "Quarkus"
    ],
    "projectType": "quarkus",
    "language": "java",
    "links": {
      "self": "/devfiles/java-quarkus/devfile.yaml"
    },
    "starterProjects": [
      "quarkus-ex"
    ]
  },
]

It would be nice if the index.json for a devfile registry could contain the list of projects in a given devfile:

do you mean projects or starterProjects (https://github.com/devfile/api/issues/42)

@kadel Thanks, good catch. I mean starter projects 🙂
I'll go update the description so it's more clear.

We discussed on today's call - we'll add the starterProjects field to the index.json of the registry. When we migrate to the new registry format, we'll make sure it's included as part of the required metadata.

starterProject still missing for java-maven

I notice that our devfile registry repo doesn't include starter project for java-maven https://github.com/devfile/registry/blob/master/stacks/java-maven/devfile.yaml, we might need to refer odo devfile registry here: https://github.com/odo-devfiles/registry/blob/master/devfiles/java-maven/devfile.yaml#L5


output full path to file when errors occur during validation

Q
A

Bug fix?
yes

New feature?
no

Breaking change?
no

Deprecations?
no

Tests added?
no

Fixed issues

License
Apache-2.0

output full path to file when errors occur during validation
for example, if a project has several index.js files with errors, previously, the jsdoc output would have ambiguous index.js errors with line numbers since only the file name without its path was listed

thinks for the help


Group length and gradient conversions with other conversions

Consider grouping the length and gradient unit conversions with the other conversions for consistency & readability. Further, consider moving all conversions to beginning or end.

Closed with commit 5652f9a.


DiscordReader ERROR:asyncio:Unclosed connector

I'm trying to use the DiscordReader to create an index for a Discord channel. I'm getting and error.
import nest_asyncio
nest_asyncio.apply()

discord_token = os.getenv("DISCORD_TOKEN")
channel_ids = [1057178784895348746]
documents = DiscordReader(discord_token=discord_token).load_data(channel_ids=channel_ids)
index = GPTListIndex(documents)
index.save_to_disk('./index/discord_index.json')

Here's the error I'm getting
INFO:discord.client:logging in using static token
logging in using static token
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/discord/http.py", line 801, in static_login
    data = await self.request(Route('GET', '/users/@me'))
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/discord/http.py", line 744, in request
    raise HTTPException(response, data)
discord.errors.HTTPException: 401 Unauthorized (error code: 0): 401: Unauthorized

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/backend/support_functions.py", line 122, in <module>

    documents = DiscordReader(discord_token=discord_token).load_data(channel_ids=channel_ids)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/llama_index/readers/discord_reader.py", line 139, in load_data
    channel_content = self._read_channel(
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/llama_index/readers/discord_reader.py", line 107, in _read_channel
    result = asyncio.get_event_loop().run_until_complete(
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/nest_asyncio.py", line 90, in run_until_complete
    return f.result()
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/asyncio/futures.py", line 201, in result
    raise self._exception.with_traceback(self._exception_tb)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/asyncio/tasks.py", line 232, in __step
    result = coro.send(None)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/llama_index/readers/discord_reader.py", line 67, in read_channel
    await client.start(discord_token)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/discord/client.py", line 777, in start
    await self.login(token)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/discord/client.py", line 612, in login
    data = await self.http.static_login(token)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/discord/http.py", line 805, in static_login
    raise LoginFailure('Improper token has been passed.') from exc
discord.errors.LoginFailure: Improper token has been passed.
ERROR:asyncio:Unclosed connector
connections: ['[(<aiohttp.client_proto.ResponseHandler object at 0x129dd25c0>,<PHONE_NUMBER>23916)]']
connector: <aiohttp.connector.TCPConnector object at 0x12e2b5000>
Unclosed connector
connections: ['[(<aiohttp.client_proto.ResponseHandler object at 0x129dd25c0>,<PHONE_NUMBER>23916)]']
connector: <aiohttp.connector.TCPConnector object at 0x12e2b5000

My understanding is the discord token is an individual user's token, not a bot token and you don't need to be the owner of the server to read the conversations. Please advise

Hey @ardaerturk Were you able to access discord API with the token you supplied?
The error message discord.errors.LoginFailure: Improper token has been passed seems to suggest that you passed a bad token.
Going to close this for now, since the issue is unrelated to LlamaIndex. Please join our discord community (https://discord.com/invite/dGcwcsnxhU) where you'd get much better support!


dependencies on swcAir projects don't link in the corresponding .swc files

I first posed this problem on the support forums, but I believe this is now a bug. The post contains a repro case, and some analysis of where the problem manifests.
The problem seems to be when constructing the project.configuration.files, the type is used as the extension of the library, instead of mapping swcAir -> swc.
I'll see if I can figure out a way to correct this, but I'll file this issue in the mean time in case anybody else has any ideas.

fixed by PR #167


Cleaning up dead servers

I just joined #social:matrix.org, and tried to have a look at synapse's logs while it was painfully trying to join.
Basically, what I saw most often was messages like these (repeated):
synapse.http.matrixfederationclient: [] {PUT-O-120641} Sending request failed to 4ray.co: PUT matrix://4ray.co/_matrix/federation/v1/send/1522796890812/: ConnectionRefusedError('Connection refused',)
synapse.http.matrixfederationclient: [] {PUT-O-121051} Sending request failed to jomo.tv: PUT matrix://jomo.tv/_matrix/federation/v1/send/1522796891206/: NoRouteError('Network is unreachable',)
synapse.http.matrixfederationclient: [] {PUT-O-121337} Sending request failed to thebeckmeyers.xyz: PUT matrix://thebeckmeyers.xyz/_matrix/federation/v1/send/1522796891492/: TimeoutError('',)
synapse.http.matrixfederationclient: [] {PUT-O-120644} Sending request failed to matrix.home.ansorg-web.de: PUT matrix://matrix.home.ansorg-web.de/_matrix/federation/v1/send/1522796890815/: DNSLookupError('no results for hostname lookup: matrix.home.ansorg-web.de',)
[a python stacktrace for BadSignatureError: Signature was forged or corrupt]
synapse.federation.federation_base: [GET-426682] Signature check failed for $15000398313789kDXuE:half-shot.uk

So all these messages have something in common: they relate to servers that appear to no longer be alive.
I don't think Matrix currently has any way of saying “this server is really dead, so let's just force-part all its users and forget about it”. I think (without having measured, again), that this kind of waiting for a big number of timeouts may be one of the things that's slowing joining a big room.
Maybe it would be reasonable to say that after all the servers currently alive in a room have seen a server for the last time 3 months ago (number to adjust, but 3 months seems enough to avoid catching accidental downtime to me), then the server is declared dead.
Dead servers would be marked as such, and no signature checking attempt would be tried for them (as another server could come in after that and re-buy the domain name), newly joining servers would be warned not to try to contact them, etc.
Does what I'm saying make sense? I'm basically thinking that:

exponential backoff is nice but won't help for newly-joining servers
at some point a server could just die and the DNS be re-bought, and then we don't want to come back a year later and assume the new owner is the same as the previous one

What do you think about it?

Let's say that the concept is acceptable, which entity would mark servers as dead? on which criterias exactly?

This is basically https://github.com/matrix-org/matrix-doc/issues/564

closing as a dup

@richvdh @ara4n Well, if the closing is wrt. #564 it's the same underlying issue that caused discovery, but I don't think these are duplicates: the proposed solution of #564 is noting in the room state which servers are available or not, and the proposed solution of this issue is to actually consider dead servers as dead.
One big difference is in the expected behaviour: I believe a dead server's users should be force-parted (kicked?) from all the rooms they are, because there is an important risk someone will just buy the domain and have access to private channels and history from the now-dead server. Which is a huge privacy risk in my opinion.
This is not solved by #564, which would fix the performance issue though (and would be a nice in-between for [0 days of downtime, 3 months of downtime]).
@maxidor I was thinking each server would remember when other servers were reachable, and when a server pushes a message “this server is dead”, other servers check it's dead to them too, and accept the message only if so and no other server sent a message “no it isn't” (to accomodate for newcomers who could not know yet whether the said server is dead or not, and will only have to trust the others).
When this message is accepted, all the users of this server for the room are force-parted.
It may be needed to elect another admin for the room (if the only ones were on the now-dead server), in which case I guess the election could happen based on some PRNG seeded on the state of the room until the last time the server was reachable (so that no server could cheat the PRNG by just stuffing some messages just before deadline expiration, and so that all servers agree on who is the new room admin).
I'd say a “this server is dead” message should be accepted if it hasn't been reachable for at least 3 months (or hasn't been reachable since joining the room for room members present for less than 3 months), and should be sent if a server hasn't been reachable for at least 3 months and 10 days (to allow for some time in case all servers don't agree exactly on the last date of reachability, and 10 days should be way more than enough).
Does what I'm saying make sense? (also, do you get why I think it's not the same thing as #564, although the performance issue is the same?)

yeah, the reason i didn't close it earlier because the proposal (boot dead servers) is different to the track retry hints idea.  a quick heuristic to do this could be to have the HS whose user created the room be responsible for tracking server health and kicking out users from dead servers, based on a state event in the room to define the kick policy. This is quite similar to previous proposals to have a state event in a room to define whether users should be kicked after being absent (as opposed to their servers being dead).

Hmm, a question in this case is, what if the HS on which the room was created is the one that dies?
I guess the best behaviour is to just close the room anyway, as its UUID could be re-taken by a replacing-with-same-DNS HS, which would cause conflicts and tears, but it sounds like a bit of a loss in the decentralization promise of matrix: if I create a room for a project, pass the admin rights to someone else then move on and someday stop maintaining my matrix server, I wouldn't expect the project to be disturbed by this.
Actually maybe that's a separate issue that would need to be discussed elsewhere, like a way of changing the server to which a room's UUID is linked? There's already the possibility of adding a #foo:bar.example.org alias to a !baz:quux.example.org room (if I remember correctly), but changing the !baz:quux.example.org to !something:else.example.org could solve this non-decentralization issue by changing “the HS whose user created the room” to “the HS on the right-hand-side of the UUID of the room”, which could then be changed with this additional proposal?


Online version is outdated

The version at https://quuz.org/webvtt/ is not using the latest version of the script.
I ran into a https://github.com/w3c/webvtt.js/commit/b2b842c569170b8ff80f88f24ac528629f7aeaa7 and determined it was a bug, but when I went to fix it… well, it'd already been fixed. :-)

cc @annevk who I believe maintain the online version

Hah, I completely forgot about that and didn't make the connection but I think I knew that about quuz once. What's up Anne? Long time no see ('07?). ;-)

Heya, long time indeed. 😊
I'm no longer maintaining it. Since it's just a script it might be possible to use GitHub Pages. I'd be happy to set up a redirect (perhaps 302 since it's GitHub Pages and not a nice stable URL) once that's in place.

I've set up https://w3c.github.io/webvtt.js/parser.html with github pages

The URL in OP now redirects, so closing.


feat!: add client observability configuration codegen

Issue #

#696
Description of changes

Note: dependent on DirectedCodegen implemented in #681 and awslabs/aws-sdk-swift#1405
Note: coupled with smithy-lang/smithy-swift#688
Adds client observability configuration codegen. Notably, config.logger is replaced with
config.telemetryProvider, which provides a default loggerProvider. See smithy-lang/smithy-swift#688
for more details.
Changes:

Breaking change: remove unused AWS default logger method

New/existing dependencies impact assessment, if applicable
No new dependencies were added to this change.

Conventional Commits

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Closed in favor of #1426