Best model is not saved properly

This is referenced in e6ecae7a74d9912bc66a14607e9e87b9a072c2ab but seems to not have been resolved. I also cannot find the commit where Scan.py was changed to remove the additions from that commit. Figured it made sense to open an issue about it.
Current implementation runs successfully:
h = ta.Scan(np.concatenate((x_train, x_dev)),
            np.concatenate((y_train, y_dev)),
            params=p,
            dataset_name='xray',
            experiment_no='1',
            model=general_model,
            grid_downsample=0.0000025,  # lots of parameters
            val_split=0.2,
            save_best_model=True)

# output
# 20 scans will take roughly 160 seconds
# Scan Finished!

But when attempting to load
from keras.models import load_model
mod = load_model('xray_1.h5')

raises
/usr/local/lib/python3.6/dist-packages/keras/models.py in load_model(filepath, custom_objects, compile)
    266         model_config = f.attrs.get('model_config')
    267         if model_config is None:
--> 268             raise ValueError('No model found in config file.')
    269         model_config = json.loads(model_config.decode('utf-8'))
    270         model = model_from_config(model_config, custom_objects=custom_objects)

ValueError: No model found in config file.

Has this been resolved already and I'm just missing something?

@x94carbone I worked this out but found that there was quite a bit of discrepancy in the way various sources said was the right way to handle it, and did not have time to test validity of the implementation I did properly so I left it out for now. There is also the case of how to really handle this beyond just the code implementation.
For example you might end up with 10-20 configurations with close to identical result. So it seems there is a need rather to add another layer of automation, where a configurable number of best results is picked, and then run again in a meaningful process that picks the best one. This leads to the question of "best" in terms of a well generalizing model vs. test performance.
The way I found the process tends to play out is something like...start with something pretty broad, and end up with some ideas on how to improve. Then narrow the boundaries, and end up with some ideas on how to improve even more. Maybe repeat this a few times, depending on the prediction task, and end up with a bunch of well performing configurations. So this will be another useful layer of process automation but involves more open questions than the one mentioned above. So I'm a little inclined to skip the "legacy" approach of picking the best model all together at least for now and go straight to figuring out an approach that appreciates the problem more.
In which case the parameter save_best_model is redundant for now.

@mikkokotila Thanks for the response! I have some ways in mind that could be used address these concerns as well. They pertain closely to my current research. What it appears you're talking about is a more insightful way of picking the best hyperparameters (HP). I have two ideas about this depending on how far you're willing to take this.
First, the first step could be exactly what Talos already does: screen different HP configurations, but then pick the top N of them. Second, of those top N models, randomly permute the test/train/dev sets and retrain/eval for each permutation. Implement some measure of model robustness such that only models that are robust to the random permutations of the data are kept and return the best overall. This also might involve a more thoughtful split of the data initially so as to avoid bias.
Second, you (we?) should consider evolutionary algorithms. This will greatly improve performance overall and could be built on top of the first step.
But anyway regarding the actual issue here I understand what you're saying. Is there any way to completely remove the option and prior package (Hyperio) to avoid confusion?

For the actual issue, I will remove all the redundant codes.

@mikkokotila Sounds good. I absolutely agree on all points. I'll comment in #18 about evolutionary algorithms and elsewhere where necessary.

The artifacts were cleaned from the current version.
The wider discussion continues in #40 so closing here.


Training gets stuck within Linear PyTorch layer

When trying to run bash scripts/perfect.sh, training gets stuck somewhere in a Linear layer of PyTorch.
For context, I'm for now only interested in yielding some result training only for SST-5, even if it is totally useless, so I've reduced some parameters to lower values to reduce computing time on a laptop with limited resources. They are as follows (defined in configs/perfect.json:
{
"task": "sst-5",
"max_seq_length": 256,
"per_device_train_batch_size": 2,
"K": 8,
"data_dir": "datasets_processed",
"output_dir": "outputs",
"model_name_or_path": "roberta-large",
"do_train": true,
"do_eval": true,
"do_predict": true,
"learning_rate": 1e-4,
"max_steps": 4,
"eval_steps": 3,
"save_steps": 3,
"data_seed": 50,
"seed": 1,
"load_best_model_at_end": true,
"metric_for_best_model": "average",
"greater_is_better": true,
"evaluation_strategy": "steps",
"save_strategy": "steps",
"save_total_limit": 1,
"overwrite_output_dir": true,
"soft_pet": true,
"soft_pet_loss": "extra_tokens",
"extra_without_original": true,
"extra_tokens_init": "random",
"soft_mask_labels_learning_rate": 1e-1,
"adapter_tune": true,
"tune_layernorms": true,
"add_layer_norm_after_adapter": false,
"add_layer_norm_before_adapter": false,
"train_in_batch": true,
"add_adapter_after_attention": false,
"add_adapter_after_feedforward": true,
"extra_embd_initializer_range": 1e-4,
"overwrite_cache": true,
"per_device_eval_batch_size": 10,
"prototypical_eval": true,
"eval_soft_pet_aggregation": "max",
"prototypical_similarity": "euc",
"token_hinge_loss": true,
"mask_position": "1"
}
The training procedure starts and gets stuck at step three out of four, first yielding the WARNING:
WARNING - datasets.arrow_dataset - Loading cached processed dataset at /home/timdadum/.cache/huggingface/datasets/json/default-9463d57d37ff152e/0.0.0/c2d554c3377ea79c7664b93dc65d0803b45e3279000f993c7bfd18937fd7f426/cache-91b7584a2265b1f5.arrow
Then, training gets stuck. Upon aborting, the following traceback is present:
Traceback (most recent call last):
File "run_clm.py", line 516, in
main()
File "run_clm.py", line 426, in main
train_result = trainer.train(resume_from_checkpoint=checkpoint)
File "/home/timdadum/anaconda3/envs/perfect/lib/python3.8/site-packages/transformers/trainer.py", line 1340, in train
self._maybe_log_save_evaluate(tr_loss, model, trial, epoch, ignore_keys_for_eval)
File "/home/timdadum/anaconda3/envs/perfect/lib/python3.8/site-packages/transformers/trainer.py", line 1445, in _maybe_log_save_evaluate
metrics = self.evaluate(ignore_keys=ignore_keys_for_eval)
File "/home/timdadum/Documents/Deep Learning Project/perfect/fewshot/third_party/trainers/trainer.py", line 133, in evaluate
output = self.eval_loop(
File "/home/timdadum/Documents/Deep Learning Project/perfect/fewshot/third_party/trainers/trainer.py", line 185, in eval_loop
metrics = self.compute_pet_metrics(eval_datasets, model, self.extra_info[metric_key_prefix])
File "/home/timdadum/Documents/Deep Learning Project/perfect/fewshot/third_party/trainers/trainer.py", line 210, in compute_pet_metrics
centroids = self._compute_per_token_train_centroids(model)
File "/home/timdadum/Documents/Deep Learning Project/perfect/fewshot/third_party/trainers/trainer.py", line 287, in _compute_per_token_train_centroids
mask_embeds.append(self.get_masks_embeds(model, batch))
File "/home/timdadum/Documents/Deep Learning Project/perfect/fewshot/third_party/trainers/trainer.py", line 267, in get_masks_embeds
hidden_states = model.roberta(input_ids=input_ids, attention_mask=attention_mask)
File "/home/timdadum/anaconda3/envs/perfect/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "/home/timdadum/Documents/Deep Learning Project/perfect/fewshot/third_party/models/roberta/modeling_roberta.py", line 868, in forward
encoder_outputs = self.encoder(
File "/home/timdadum/anaconda3/envs/perfect/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "/home/timdadum/Documents/Deep Learning Project/perfect/fewshot/third_party/models/roberta/modeling_roberta.py", line 541, in forward
layer_outputs = layer_module(
File "/home/timdadum/anaconda3/envs/perfect/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "/home/timdadum/Documents/Deep Learning Project/perfect/fewshot/third_party/models/roberta/modeling_roberta.py", line 468, in forward
layer_output = apply_chunking_to_forward(
File "/home/timdadum/anaconda3/envs/perfect/lib/python3.8/site-packages/transformers/modeling_utils.py", line 2196, in apply_chunking_to_forward
return forward_fn(*input_tensors)
File "/home/timdadum/Documents/Deep Learning Project/perfect/fewshot/third_party/models/roberta/modeling_roberta.py", line 481, in feed_forward_chunk
layer_output = self.output(intermediate_output, attention_output)
File "/home/timdadum/anaconda3/envs/perfect/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "/home/timdadum/Documents/Deep Learning Project/perfect/fewshot/third_party/models/roberta/modeling_roberta.py", line 393, in forward
hidden_states = self.dense(hidden_states)
File "/home/timdadum/anaconda3/envs/perfect/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "/home/timdadum/anaconda3/envs/perfect/lib/python3.8/site-packages/torch/nn/modules/linear.py", line 114, in forward
return F.linear(input, self.weight, self.bias)
KeyboardInterrupt
Which seems to suggest that somehow PyTorch can't comprehend the data processing done in Arrow. I'm not sure how to fix this, so any help would be welcome!

SOLVED: Running on Google Cloud services fixed it.- Probably still some memory issue of some sorts.


implement 'get nodes command'

this command list all nodes (object) in a given repository .
Note: listing attributes should include the dataset from which the node resides.

Each 'node' in a JSON-LD document (i.e. dataset) consists  of  a type(@type) or types (value can be list), id (@id) and a collection of zero or more attributes (edge) and their values. Also, each node belongs to a graph in the document. This can be the default graph or a named graph.  I would like to see the get nodes command produce the following columnar output

ID
Type
Property
Property Value  (may need to display only first N characters if longer than some determined reasonable length for display)
Graph name ('default' if in default graph)

should specify dataset (--dataset ).
In the future perhaps considering displaying all nodes in a repository.

would like implementation that searches json-ld document and produces matching results to leverage channels (and perhaps go routine) similar to what was implemented in  impl/file/repo/index.go for fileRepositoryIndex.scan() which sends results back tthrough a channel to the caller. this prevents allocating return arrays of unbounded size and allows returning of matching results as they are found to the caller in addition to responding to cancelling of context if performed by caller.


Multiple Ding model creation and number of past stimulation

Subject of the enhancement
This is a refactor that can be done.
Proposed enhancement
model = DingModel(n_stim_prev: int, ....)
model.nb_stim_prev --> val
MultiDing(....).to_models --> [model1, model2,...]

maybe it's easier to have:
model.set_all_pulse_apparition_time([.....])


WIP added net7.0-windows10.0.19041.0 target to uno.winui and winui

Added net7.0-windows10.0.19041.0 Uno.WinUI Target and fixed some duplicated code in the Project (Probably because of a merge)

I'll wait with merging until the build is fixed.

So the Android Builds should now work again. (Not necessary Xamarin.Android nuget packages were removed and the remaining fixed.


Compiler passes section collab

Updating the section on compiler passes to collaborate with @jbobba and others on how to better present this info.

Adding @rkimballn1  to review the latest changes on 165500d5d5341c6ac803fca3f3a6536f3074e10e since these are from the presentation, and I'm unsure if they were affected by the new backend API changes.

@jbobba and I worked on this via email, so he requested to be removed from PR


Fix conflicting docs for /tekton/steps API Compatibility.

Changes
Previously the developer docs listed that /tekton/steps was in scope for
the API compatibility policy, where-as the user
docs
did not say it was in scope.
After discussion in
PR #4352,
we have come to agreement that this is not in scope.
While this change in itself is not a breaking change, this opens the
door for breaking changes (such as #4352). Our stance is that users
should use the $(steps.step-<step-name>.exitCode.path)
syntax instead to ensure compatibility.

/kind documentation
Submitter Checklist
As the author of this PR, please check off the items in this checklist:

[x] Docs included if any changes are user facing
[x] Tests included if any functionality added or changed
[x] Follows the commit message standard
[x] Meets the Tekton contributor standards (including
functionality, content, code)
[x] Release notes block below has been filled in or deleted (only if no user facing changes)

Release Notes
action required: Users should not assume the directory/file structure of /tekton/steps. Use $(steps.step-<step-name>.exitCode.path) instead.

/assign @bobcatfish @pritidesai

/test pull-tekton-pipeline-alpha-integration-tests

/test pull-tekton-pipeline-alpha-integration-tests

Thanks for the discussion and update @pritidesai @wlynch !
/lgtm


make install overwrites /usr/include/string.h

It seems like include/ak/*.h are directly copied to /usr/include.

Hi @emersion
Good catch, make file must be updated like cglm's build files. The current active branch is gltf-json.
I have written new json library (https://github.com/recp/json) to parse glTF and other json stuff. After that I've dropped old JSON parser and used the new one. It is almost finished, almost.
Also AssetKit is based on COLLADA and glTF. I'm trying to make it more simpler by dropping complexities that comes from COLLADA...
Any contribution is welcome 🤗

@emersion this must be fixed at https://github.com/recp/AssetKit/commit/665645ec2ca6874fa6cf09bc408e4834cec905b5
Also AssetKit is provides CMake build files.
Thanks!

Nice, thanks!


Proposal: implementation strategy for ACME RFC8555 in x/crypto/acme

The goal of this proposal is to agree on the implementation details of RFC8555 in #21081.
The acme package currently implements an approximation of draft-02, aka "ACMEv1 API".
The RFC8555 is the final version of the ACME protocol, aka "ACMEv2 API", based on draft-18. Let's Encrypt announced End of Life of their ACMEv1 in https://community.letsencrypt.org/t/end-of-life-plan-for-acmev1/88430 where first deadline is Nov 2019 past which no new account registrations will be allowed.
Note that we have 2 packages in scope:

https://golang.org/x/crypto/acme: this is a barebones low level client essentially mirroring the ACME API with some helper functions;
https://golang.org/x/crypto/acme/autocert: it is based on the former, offering to users users completely automated workflow of cert issuance and renewals.

While this issue is mostly concerned with the acme package, autocert package will also be impacted in different ways based on this proposal resolution although it should be a relatively easy set of changes since most of its functionality is automated and unexported.
There are currently two conflicting proposals. I will list pros & cons for each one as I see them.
1. Replace golang.org/x/crypto/acme with a strict RFC8555 implementation, essentially removing all remnants of draft-02.
An example of this is https://golang.org/cl/86635.
Pros:

removes draft-02 bits which aren't used or replaced with alternatives in the RFC
already have a CL although it needs to catch up with some later additions
to the current acme package and split up in smaller CLs to make it possible
for a reasonable human review
clean slate: can completely rewrite all of acme and the Client API
to possibly improve although I don't have concrete shiny examples

Cons:

breaks all existing users
makes acme and autocert packages incompatible with other ACME CAs which haven't
implemented RFC8555 yet; at the time of writing:

BuyPass: only ACMEv1 in production
Entrust: seems to provide only ACMEv1 endpoint
GlobalSign: unclear but I suspect v1, although I've heard they might be working on v2; totally speculative
Venafi: judging from their docs it's a v1 although based on draft-06; certainly not RFC anyway
EJBCA: draft-12, so it isn't actually too far from the RFC
there are a couple private CAs that I know of; they're still at draft-02

2. Add RFC855 support to the existing acme package without breaking its users and their issuance using ACME CAs
The major exported changes would be the following:

new acme.Client methods CreateOrder and FinalizeOrder to implement order-based issuance flow which is preferred in the RFC, as opposed to the pre-authorization flow using the existing Authorize method; the pre-authorization flow is optional in RFC and in fact Let's Encrypt does not provide it in their ACMEv2 but some other CAs do
new exported structs such as Order to accomodate the new order-based issuance flow and new fields in the existing exported structs such as those in acme.Directory in https://go-review.googlesource.com/c/crypto/+/182937/2/acme/types.go; I expect them to have valid zero values or be populated by the acme client accordingly
JWS request signature format and requirement of kid and url fields; although most changes
should be transparent, they may require some new acme.Client methods to accommodate the new JWS/JWK formats

I do not expect for the existing exported methods of acme.Client to be removed or their signatures changed. Their implementation will branch according to the endpoint in use: ACMEv1 or ACMEv2. This is possible because the Client performs directory fetch automatically upon first network round-trip for any client call whenever required using its Discover method. It can thus identify which version of ACME protocol is in use at the right time.
Later on, we can amend the acme client and remove ACMEv1 support as more CAs catch up with the RFC compliance.
Pros:

doesn't break existing users, or at least it shouldn't - that's the idea
keeps working with the CAs mentioned above which haven't migrated to the RFC yet

Cons

legacy baggage to support non-RFC compliant CAs until the majority of them are,
which most likely means more complicated code until the support for non-compliant CAs
is dropped completely

A non-exhaustive list of alternatives I've been thinking of:

Use go mod and have two versions of /x/crypto since acme is a sub directory. I don't belive
acme changes alone justify this.
Move current acme,autocert packages into something like /v1 sub directory and replace acme pkg with an RFC8555-rewrite. We now have go mod; this feels like a step backwards.
Add RFC855-only package into a /v2 and at some point replace the current acme with it. Same
as before: feels like a step backwards considering go mod.
Fork current acme outside for /x/crypto and replace x/crypto/acme with RFC8555 to let those
who's still stuck with non-RFC compliant CAs still run. This fork will unlikely get future
improvements unless someone will have time to backport the changes.
Move the whole /x/crypto/acme/... somewhere else outside of /x/crypto. This would've allowed go mod versioning. It's probably too big of a change and a bit ouf of scope here.

Personally, I'm in favour of (2), i.e. add RFC support to the existing acme client and keep it working with all major ACME CAs out there until reasonably possible without too much of an overhead.

Why not both? In the go mod world, it's been formalized that if you make an incompatible revision of a library, it has to live at v2/ forever. So, we could commit new code that supports RFC 8555 under crypto/acme/v2, and keep the v0 library under crypto/acme. That would allow current code to keep working, as well as providing an upgrade path ASAP.
This related to https://github.com/golang/go/issues/21324 - with go mod support becoming mainstream in 1.13, I think it's valuable for the x/all libraries to show confidence in the versioning plan that Go recommends.

Another trouble with /v1 and /v2 I forgot to mention is autocert: it would need to import both and use one or the other based on the ACME version. It complicates autocert code too much imho.
Alternatively, you'd need to make separate /v2/autocert.

Regarding this con in the first plan:

makes acme and autocert packages incompatible with other ACME CAs which haven't
implemented RFC8555 yet; at the time of writing:

has there been positive confirmation that any of these CA's implementation of "ACME v1" function with the existing codebase?
I don't think it can be taken on faith or based on limited public documentation. There's no specification for ACME v1 and no single draft that can be identified as a closest match. I don't think losing compatibility with these CAs should be considered as a con unless it can be confirmed they were compatible to begin with.

I just tried now with the first in the list using a small cmd line tool which calls acme.Client directly.
$ acme reg -gen -accept -d https://api.buypass.com/acme/directory mailto:XXX
$ acme whoami
URI:		 https://api.buypass.com/acme/acct/1TYYjD4k1-6bNw
Key:		 ZZZ/.config/acme/account.key
Contact:	 mailto:XXX
Terms:		 https://api.buypass.com/acme/terms/750
Accepted:	 yes

$ acme cert -d https://api.buypass.com/acme/directory -expiry 24h -dns one.acmetest.bots.run
Add a TXT record for _acme-challenge.one.acmetest.bots.run with the value "aya3lx3gaVWF31c61-i_lwX_tkmnjqrpQyNuf3vFn74" and press enter after it has propagated.
<ENTER>
cert url: https://api.buypass.com/acme/cert/K1xHK-DunMk

$ openssl x509 -noout -serial -issuer -dates -subject -in ~/.config/acme/one.acmetest.bots.run.crt
serial=10619A17D62C509CF49B
issuer=C = NO, O = Buypass AS-983163327, CN = Buypass Class 2 CA 5
notBefore=Jul 24 18:38:25 2019 GMT
notAfter=Jul 25 21:59:00 2019 GMT
subject=CN = one.acmetest.bots.run

Can try with some others but it's missing the point a bit. The list I composed is just a sample. Plus, some private CAs like I mentioned.

@x1ddos: Why is it missing the point? Your con says:

"makes acme and autocert packages incompatible with other ACME CAs which haven't implemented RFC8555 yet".

I think establishing whether it was compatible or not to begin with is important in understanding the cost of breaking the compatibility. Since there is no specification its very difficult to ascertain compatibility without experimentation. Entrust's documentation specifically assumes Certbot, and I suspect other pre-RFC 8555 CAs used the "it works with Certbot so it's ACME v1 compatible" approach to interop.
I'm convinced that Buypass' pre-RFC 8555 ACME endpoint is compatible but are the others? Breaking compatibility with 1 CA is a different level of compromise than breaking compatibility with 5+.

Why is it missing the point?

realistically, you'll never get a confirmation from all of them and they won't upgrade to be RFC compliant in 2 months anyway
there are always hacks. Many have tried to reach a compliance in HTTP servers with browser vendors or have a clean OAuth2 client implementation. Never worked. I doubt it'll be that much different with RFC8555.
so, to extend the previous point, if the package will indeed contain hacks, I'd rather it worked with 5+ CAs than 1 or 2. Dropping everything just to be RFC compliant and nothing else doesn't really help it and would certainly make it much harder to cover potential special cases in 5+ CAs in the future which haven't reached RFC.

realistically, you'll never get a confirmation from all of them

Indeed. So the compatibility being lost is primarily hypothetical :+1:

Many have tried to reach a compliance in HTTP servers with browser vendors or have a clean OAuth2 client implementation. Never worked. I doubt it'll be that much different with RFC8555.

Perhaps, but like with HTTP for RFC 8555 there is standards text that can be used to mediate compliance. There is no such thing for ACME v1. There is no way to determine if a change requested by a server operator for compliance is a one-off request for a specific implementation, or a deviation from a standard that would affect multiple implementations without experimentation.

so, to extend the previous point, if the package will indeed contain hacks, I'd rather it worked with 5+ CAs than 1 or 2

It seems like we're in agreement then: to know how to evaluate the hacks required means needing to understand the CAs that are compatible with this package now, and how many will be compatible based on proposed changes to the package.

@cpu this is a nice theory. Unfortunately, practice tells me a different story which is why I expressed my preference towards option 2.

@cpu I think the point here is not to say that the library shouldn't upgrade to ACMEv2, it is to say, should it do so in a way that breaks people with no notice? I would go so far to say if a package broke me with no notice on a major thing like this I would have trust issues with the package.
With that in mind, it does not seem unreasonable to me to try to maintain backward compatibility with the current behavior for some period of time.
To be clear personally, do not like the idea of a merged codebase as is represented in option #2. With that said, it does sound like it doesn't break anyone (which I think is important) and gets Let's Encrypt ACMEv2 in the library in time for their migration from their "past ACME implementation" to the 8555 compatible one.
I would like to recommend that if that approach is taken it comes with a statement that supports prior implementation be dropped at some specific and notified point of time so a cleaner and more maintainable library is in place long term.

In Go, there is always an implicit compatibility target, which is what currently works.
I agree that there is no such a thing as ACMEv1 to target, so now that there is a specified ACME protocol we shouldn't spend any more effort chasing compatibility with pre-RFC CAs. But that's different from changing behavior in such a way that breaks what used to work before the change.
As reflected by both the the Go 1 Compatibility Promise and by modules' semantic versioning (although this package is not bound by either), the ecosystem really values this kind of stability.
That doesn't mean we need to keep supporting "ACMEv1", but precisely the servers that already work, because that's what users will have tested and developed an expectation for.
Considering this, option 2 here is generally preferable, if the API complexity cost is moderate. It looks like there wouldn't be more than a couple of v1-only and v2-only methods, which looks acceptable. I like the Discover based autodetection.
As an implementation strategy, I'd like to see v2 code in separate files as much as possible, even if it means more code duplication. That should make it easier to rip out v1 eventually, and will ensure the v2 implementation is good enough to stand on its own.

Ah, about using modules versioning for this: there are two ways to do it, either bumping all of x/crypto to v2, causing an import path change that is not necessary for most packages, or splitting x/crypto/acme into its own module, and then bumping that.
Nested modules are a lot of complexity, they carve a hole in the parent module, and interact weirdly with it. To maintain compatibility we'd probably have to keep the current acme package in x/crypto in its current state, but we'd probably not want to have two different autocert versions, and I am not sure we can redirect from one to the other. I also worry that v2 would be less discoverable than v0 because the latter would be what projects already have in their go.mod.
It's a judgement call, but I think this complexity is more than that of supporting both protocols in the same package.

Just chiming in that we are starting to see failures from this now:
https://github.com/Velocidex/velociraptor/issues/120
This message https://community.letsencrypt.org/t/important-notice-to-acme-client-developers-regarding-acme-v1-deprecation/100795 points me to a list of platforms that support this new protocol here https://github.com/letsencrypt/website/blob/master/data/clients.json and autocert seems to be the only golang option and it does not support ACMEv2.
If we are starting to see rolling outages in prod it would be nice to have this fixed ASAP (or at least have some supported alternative for the time being).

Ah just saw 0e8c3a9068a2fb325459490d2ce7baa4ffe2d00f thanks for that! We will rebuild our binaries at that commit.
Thanks for the prompt fixes. Maybe this issue can be closed?


Inline parameter descriptions yield strange results

These comments:
/// Initializing constructor
connection_params(
        boost::string_view username, ///< Username
        boost::string_view password, ///< Possibly empty password
        boost::string_view db = "",   ///< Database to use, or empty string for no database.
        collation connection_col = collation::utf8_general_ci, ///< The default character set and collation for the connection.
        const ssl_options& opts = ssl_options() ///< The TLS options to use with this connection.
    )

Yield this:

Workaround: use \param (that works fine)

A javadoc on each parameter is not a feature of doxygen


The examples does not work

The example for CosmosDB does not work since the API has changed. For instance ConnectionStringSetting aren't part of the call signature.
And I got this error when running it (I had to jump through some hoops to run it as well because of execution policy):
[2023-03-29T14:21:28.265Z] Executed 'HttpExample' (Failed, Id=de6a3fb1-1ace-478b-94a9-1ea6117dd507, Duration=129ms)
[2023-03-29T14:21:28.266Z] System.Private.CoreLib: Exception while executing function: HttpExample. Microsoft.Azure.WebJobs.Host: Exception binding parameter 'documentsOut'. Microsoft.Azure.WebJobs.Extensions.CosmosDB: Cosmos DB connection configuration 'CosmosDB' does not exist. Make sure that it is a defined App Setting.

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 62965f3a-d852-cbe8-3d01-a5126a16c660
Version Independent ID: 2af2bfc6-bf21-08c6-a874-467781a4af90
Content: Connect Azure Functions to Azure Cosmos DB using Visual Studio Code
Content Source: articles/azure-functions/functions-add-output-binding-cosmos-db-vs-code.md
Service: azure-functions
GitHub Login: @ggailey777
Microsoft Alias: glenga

@ztamizzen
Thanks for your feedback! We will investigate and update as appropriate.


Training with unknown/variable image sizes

I am trying to train the model with variable input image sizes. However, by setting:
input_size = None in main(), I get:
Traceback (most recent call last):
  File "/.../tensorflow-deeplab-resnet/train.py", line 254, in <module>
    main()
  File "/.../tensorflow-deeplab-resnet/train.py", line 142, in main
    image_batch, label_batch = reader.dequeue(args.batch_size)
  File ".../tensorflow-deeplab-resnet/deeplab_resnet/image_reader.py", line 179, in dequeue
    image_batch, label_batch = tf.train.batch([self.image, self.label], num_elements)
  File "/.../python2.7/site-packages/tensorflow/python/training/input.py", line 922, in batch
    name=name)
  File "/.../python2.7/site-packages/tensorflow/python/training/input.py", line 707, in _batch
    capacity=capacity, dtypes=types, shapes=shapes, shared_name=shared_name)
  File "/.../python2.7/site-packages/tensorflow/python/ops/data_flow_ops.py", line 688, in __init__
    shapes = _as_shape_list(shapes, dtypes)
  File "/.../python2.7/site-packages/tensorflow/python/ops/data_flow_ops.py", line 76, in _as_shape_list
    raise ValueError("All shapes must be fully defined: %s" % shapes)
ValueError: All shapes must be fully defined: [TensorShape([Dimension(None), Dimension(None), Dimension(3)]), TensorShape([Dimension(None), Dimension(None), Dimension(1)])]

It seems the current approach, of defining a batch in dequeue() in image_reader.py, is incompatible with variable input image dimensions. Is it currently possible to train without fixed image dims?

i think you can use tf.shape outputs coupled with the set_shape function to achieve that. You'd better consult the tensorflow resources for that; this issue is out of scope of this repo.


The bot sends the welcome message only when I start writing something (WebChat V4)

Github issues for C#
Sample information

Sample type: sample
Sample language: dotnetcore
Sample name: Dispatch Bot Sample

Describe the bug
When I want to write in the webchat with the bot he does not send the welcome message to me. Only when I start to write something, he sends me the welcome message + the answer
To Reproduce
Follow the Tutorial from the Dispatch Bot Template
Add the Webchat in the Channels
Open the Bot via the Webchat (The iFrame Snippet from the Webchat Channel)
= He isnt instantly sending the welcome message
Expected behavior
the bot should directly send the Welcome message
Screenshots
In this screenshot, i changed the normal text-output with an Adaptive Card

[bug]

@danieldangler, the welcome message displays as you begin to type or after you have hit the send button?

@danieldangler, the welcome message displays as you begin to type or after you have hit the send button?

@stevkan only after i press the send button

@stevkan Any updates on this?

@danieldangler, thank you for your patience. I've tried replicating your issue but am not able to. Your method of implementing an adaptive card as a welcome response doesn't quite match the method demonstrated in the sample shown here.
Could update your code to match, test again, and let me know if this affects the timing of the welcome message displaying?

@danieldangler, can you also verify for me which channel you are using? And, do you encounter the same issue if you test other channels, namely Emulator and customizable Web Chat, if you have that one setup.
In testing the iframe Web Chat version, any welcome message is not sending upon connection which, I believe, is expected behavior. I will need to double check that to be certain.

Closing due to inactivity.


Upgrade grpc version to v1.32.0

Description
Upgrade grpc version to v1.32.0 and fix breaking api changes.
Issue reference
https://github.com/dapr/dapr/issues/2101
Checklist
Please make sure you've  completed the relevant tasks for this PR, out of the following list:

[x] Code compiles correctly
[ ] Created/updated tests
[x] Unit tests passing
[ ] End-to-end tests passing
[ ] Extended the documentation / Created issue in the https://github.com/dapr/docs/ repo: dapr/docs#[issue number]
[ ] Specification has been updated / Created issue in the https://github.com/dapr/docs/ repo: dapr/docs#[issue number]
[ ] Provided sample for the feature / Created issue in the https://github.com/dapr/docs/ repo: dapr/docs#[issue number]

@yaron2 @youngbupark
This is  Last PR for grpc version upgrade.

Faild for lint checking:
run golangci-lint
Running [/home/runner/golangci-lint-1.31.0-linux-amd64/golangci-lint run --out-format=github-actions] in [] ...
Error: SA1019: package github.com/golang/protobuf/proto is deprecated: Use the "google.golang.org/protobuf/proto" package instead.  (staticcheck)
Error: SA1019: package github.com/golang/protobuf/proto is deprecated: Use the "google.golang.org/protobuf/proto" package instead.  (staticcheck)
Error: issues found
Ran golangci-lint in 110598ms

Faild for lint checking:
run golangci-lint
Running [/home/runner/golangci-lint-1.31.0-linux-amd64/golangci-lint run --out-format=github-actions] in [] ...
Error: SA1019: package github.com/golang/protobuf/proto is deprecated: Use the "google.golang.org/protobuf/proto" package instead. (staticcheck)
Error: SA1019: package github.com/golang/protobuf/proto is deprecated: Use the "google.golang.org/protobuf/proto" package instead. (staticcheck)
Error: issues found
Ran golangci-lint in 110598ms

Can you replace the proto package?

/ok-to-perf

Faild for lint checking:
run golangci-lint
Running [/home/runner/golangci-lint-1.31.0-linux-amd64/golangci-lint run --out-format=github-actions] in [] ...
Error: SA1019: package github.com/golang/protobuf/proto is deprecated: Use the "google.golang.org/protobuf/proto" package instead. (staticcheck)
Error: SA1019: package github.com/golang/protobuf/proto is deprecated: Use the "google.golang.org/protobuf/proto" package instead. (staticcheck)
Error: issues found
Ran golangci-lint in 110598ms

Can you replace the proto package?

Should we remove this hint check item SA1019? As discussed here:
https://github.com/golang/protobuf/issues/1077

Faild for lint checking:
run golangci-lint
Running [/home/runner/golangci-lint-1.31.0-linux-amd64/golangci-lint run --out-format=github-actions] in [] ...
Error: SA1019: package github.com/golang/protobuf/proto is deprecated: Use the "google.golang.org/protobuf/proto" package instead. (staticcheck)
Error: SA1019: package github.com/golang/protobuf/proto is deprecated: Use the "google.golang.org/protobuf/proto" package instead. (staticcheck)
Error: issues found
Ran golangci-lint in 110598ms

Can you replace the proto package?

Should we remove this hint check item SA1019? As discussed here:
golang/protobuf#1077

I think we should update to the new package.
See here: https://github.com/golang/protobuf/issues/1077#issuecomment-612563120

Faild for lint checking:
run golangci-lint
Running [/home/runner/golangci-lint-1.31.0-linux-amd64/golangci-lint run --out-format=github-actions] in [] ...
Error: SA1019: package github.com/golang/protobuf/proto is deprecated: Use the "google.golang.org/protobuf/proto" package instead. (staticcheck)
Error: SA1019: package github.com/golang/protobuf/proto is deprecated: Use the "google.golang.org/protobuf/proto" package instead. (staticcheck)
Error: issues found
Ran golangci-lint in 110598ms

Can you replace the proto package?

Should we remove this hint check item SA1019? As discussed here:
golang/protobuf#1077

Yes

OK. Please help to remove this hint checking. Now this PR is blocked by this checking fail.

Faild for lint checking:
run golangci-lint
Running [/home/runner/golangci-lint-1.31.0-linux-amd64/golangci-lint run --out-format=github-actions] in [] ...
Error: SA1019: package github.com/golang/protobuf/proto is deprecated: Use the "google.golang.org/protobuf/proto" package instead. (staticcheck)
Error: SA1019: package github.com/golang/protobuf/proto is deprecated: Use the "google.golang.org/protobuf/proto" package instead. (staticcheck)
Error: issues found
Ran golangci-lint in 110598ms

Can you replace the proto package?

Should we remove this hint check item SA1019? As discussed here:
golang/protobuf#1077

Yes

OK. Please help to remove this hint checking. Now this PR is blocked by this checking fail.

https://github.com/dapr/dapr/pull/2279

/ok-to-test

Test failed due to Helm download timeout..

/ok-to-test

Thanks for your great work on this @skyao


Middleware doesn't like URLs with trailing slashes.

The "siteurl" field doesn't like it when you pass in a URL like this:
sqsMiddleware.set('siteurl', 'https://some-site.squarespace.com/');
This one's easy enough. I'll fix this issue.

Fixed after merging your PR!


D19 - Explain BPI Subject Session

D19 states: A BPI Subject Account SHOULD be re-encrypted after every BPI Subject session to one or more cryptographic keys only known to the BPI Subject Account holder.
BPI Subject Session is not explained. Implicit to this requirement is that the BPI subject account is explicitly decrypted at the beginning of the session, and available for the whole duration of the session. Would you please provide clarity on the lifecycle of the session?

@atoulme ... how about this? cc @chaals @humbitious
[D19] A BPI Subject Account SHOULD be encrypted when not in active use by the BPI Subject that owns the BPI Subject Account.

Closing. Addressed with merged PR https://github.com/eea-oasis/baseline-standard/pull/173


Merge sync c63f91b

Summary
Sync to latest upstream #c63f91b

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.1 out of 5 committers have signed the CLA.:white_check_mark: dkstyle:x: ryota-murakami:x: jcmdonald:x: chenxsan:x: snitin315You have signed the CLA already but the status is still pending? Let us recheck it.


.Net Add support for quality and style for OpenAI image generation

Draft PR for visibility only

@microsoft-github-policy-service agree


documentation for Finding GCD using Euclidean algorithm || math

🚀 Feature
(A clear and concise description of what the feature is.)
Have you read the Contributing Guidelines on Pull Requests?
yes
Motivation
GCD using Euclidean algorithm
Pitch
GCD using Euclidean algorithm

@HarshCasper  @iamrajiv  Can you please assign me this issue


Tips/docs to identify no connectivity between publisher and subscriber

I was trying simple publisher/subscriber example between 2 servers in aws.
Subscriber listening on ip "<IP_ADDRESS>" and publisher sending on public ip of same.
I have established that connectivity between those instances is present by sending texts on udp through netcat on same port, Which worked as expected. Initially i thought this might some os/config related issue netcat on udp should also not be working in that case. Any hint to identify the problem here.

Hi issue can be reproduced with ami id CentOS-6.5-GA-03.3-f4325b48-37b0-405a-9847-236c64622e3e-ami-6be4dc02.2 (ami-8997afe0).
Settings included iptables off.
selinux off
with above nc was able to reach to server on udp but Aeron was generating not connected errors.
I had moved to different ami where things aeron connectivity is working for me, But it would be nice to know steps to identify/debug the issue in general.

Support can be purchased for Aeron.


Use more ascii characters for more efficient uft-8 encoding

If you want your base256 encoding to be as efficient as possible you should use every available printable ascii character. In UTF-8 Ascii character is represented in 1 byte while anything beyond ascii is 2 bytes, so using all 94 (without space) or 95 (with space) and only using Unicode characters in the alphabet after the from 95-255 or 96-255..

Yes, using more ASCII characters would be more efficient (1 byte vs 2 bytes). However, I decided against it for 2 reasons:

Just small data is typically represented in Base256, e.g. 8/16/32/64/128 bytes for passwords/hashes/etc. Therefore, efficiency is not top priority.
Please note that non-terminal characters have been used only for Base256 to support double-clicking for copy&paste. With terminal characters (+-/*.,%&=?...) this would be much more complicated and error-prone. Just think of period or commata at the end.


Unknown "form_javascript" function

I am using the 4.0.2 version with Symfony 3.0.4 and i get this error:
Unknown "form_javascript" function in IvoryCKEditorBundle:Form:javascript.html.twig at line 3.

How do you render your form? DO you use the async feature? Basically, I need more informations in order to understand what's goind on :)

No, i am using the default config.
In fact the error comes in another form where i dont use the CKEditor field.
This is my config section:
ivory_ck_editor:     default_config: minimal     plugins:         uploadimage:             path:     "/ckeditor/plugins/uploadimage/"             filename: "plugin.js"         widget:             path:     "/ckeditor/plugins/widget/"             filename: "plugin.js"         image2:             path:     "/ckeditor/plugins/image2/"             filename: "plugin.js"         uploadwidget:             path:     "/ckeditor/plugins/uploadwidget/"             filename: "plugin.js"         clipboard:             path:     "/ckeditor/plugins/clipboard/"             filename: "plugin.js"         lineutils:             path:     "/ckeditor/plugins/lineutils/"             filename: "plugin.js"         filetools:             path:     "/ckeditor/plugins/filetools/"             filename: "plugin.js"         notificationaggregator:             path:     "/ckeditor/plugins/notificationaggregator/"             filename: "plugin.js"         notification:             path:     "/ckeditor/plugins/notification/"             filename: "plugin.js"     configs:         minimal:             skin: "flat,/ckeditor/skins/flat/"             extraPlugins:   'uploadimage,image2'

Ok thanks for you're quick feedback :) I got the problem but I need to find the good way to fix it... The problem is the ckeditor_javascript block will never be used by your code but it will be compiled by twig anyway (regardless it is used or not)... So, since the missing function is shipped in a third party bundle, twig is not able to compile the template and end up with your error...

 Just need to add the following to your composer.json:
{
    "require": {
        "egeloen/ckeditor-bundle": "dev-form-javascript"
    }
}

It works!

@plozmun Your issue has been fixed in the referenced PR and is part of the new release.


allow dry eval of URL configs

What does the code in this PR do / what does it improve?
Allow evaluation of URL configs outside of context.
Sometimes, it's very useful to test the output of url-configs outside of a context. Just to sanity check that we are getting the right values. Until this moment, I always first changed the plugin, than checked that the plugin was giving me the correct value I was asking for. However, apart from the .plugin attributes, the URL config can be used in a broader sense than only for plugins. So I wrote a very small class method to do just that.
I had to scratch my head that it was so simple (of course that's @jmosbacher 's work 🎉) but probably still useful to add for it's nice to have since I'm sure people will also be running into the question "how do I know I've written some URL config that makes sense?"
Can you give a minimal working example (or illustrate with a figure)?
Please see the example also written in the docstring/test:
plugin_url = 'cmt://electron_drift_velocity?run_id=plugin.run_id&version=v3'
self.st.set_config({'test_config': plugin_url})
p = self.st.get_single_plugin(nt_test_run_id, 'test_data')
correct_val = p.test_config

# We can also get it from one of these methods
dry_val1 = straxen.URLConfig.evaluate_dry(
    f'cmt://electron_drift_velocity?run_id={nt_test_run_id}&version=v3')
dry_val2 = straxen.URLConfig.evaluate_dry(
    f'cmt://electron_drift_velocity?version=v3', run_id=nt_test_run_id)

# All methods should yield the same
assert correct_val == dry_val1 == dry_val2

Coverage increased (+0.03%) to 93.901% when pulling bc0547a187d64e5c73fbe7b7c1e3d8ff14762bf6 on dry_url_config into 48e86bcd2e659755958f3370faf78f146477d656 on master.

Nice! BTW this was actually addressed in #827 with the URLConfig.eval class method. But I am completely fine with adding this stop-gap until we decide what parts of #827 to merge post sr0.

@jmosbacher Thanks, should of course have consulted that PR first.


Tracer login

Adds tracer login button, sign in form, calls backend to check credentials, and then shows create psa button if correct

Perfect! Thanks!


Adding postbuild for automatic building

Hello,
I wanted to know if a pull request that added https://github.com/exogen/postinstall-build would be accepted ?
This will allow the installation of the project from a different source than npm (for example git, I also use it with file type of packages) without having to maintain a different branch with the compiled source.

Hey @bilby91 ,
postinstall-build will be a dependency for all users who install sequelize-typescript then, despite of they probably doesn't need it. Probably we can find another solution. A branch called build for example, that is created automatically on every push to the master. Maybe there is a github plugin that achieves this? This would fit your requirements very well, I think? Without having an additional dependency.

@bilby91 Any thoughts on this?

@RobinBuschmann Yep, I guess that is the only drawback. The dependency will be shipped always :(.
Yep, I guess that we can have a master-compiled branch that can be automatically generated by CI after a merge on master. That should work for master.
The only "problem" will be if people want to install a particular branch that is not master.


Messages prop for Reply Messages

messages="[
  {
    _id: 7890,
    content: 'message 1',
    sender_id: 1234,
    username: 'John Doe',
    date: '13 November',
    timestamp: '10:20',
    system: false,
    saved: true,
    distributed: true,
    seen: true,
    disable_actions: false,
    disable_reactions: false,
    file: {
      name: 'My File',
      size: 67351,
      type: 'png',
      audio: true,
      duration: 14.4,
      url: 'https://firebasestorage.googleapis.com/...'
    },
    reactions: {
      wink: [
        1234, // USER_ID
        4321
      ],
      laughing: [
        1234
      ]
    }
  }
]"

What is the attribute to say that a message is a reply of another message? I am using the reply action but I only see the message that I send and not the one that I am replying.

I'm using webSocket to send and receive messages, and in the websocket i send the object with the same structure of the example.

Indeed it's missing in the docs. Will add it:
messages="[
  {
    _id: 7890,
    content: 'message 1',
    sender_id: 1234,
    ...,
    replyMessage: {
      _id: 5543,
      content: 'reply',
      sender_id: 4321,
      file: ...
    }
  }
]"

This is what I'm sending to DB:
async sendMessage({ content, roomId, file, replyMessage }) {
  if (replyMessage) {
    message.replyMessage = {
      _id: replyMessage._id,
      content: replyMessage.content,
      sender_id: replyMessage.sender_id
    }

    if (replyMessage.file) {
      message.replyMessage.file = replyMessage.file
    }
  }
}

It works!, But I have another question, what is the correct way to send audio messages? because I am testing and I do the following:
 async sendMessage({ roomId, content, file, replyMessage }) {
            console.log("******************* Send Message *******************");
            console.log("roomId", roomId);
            console.log("content", content);
            console.log("file", file);
            console.log("replyMessage", replyMessage);
            console.log("****************************************************");

            const messages = this.messages;

            let newMessage = {
              _id: getUUID(),
              content: content,
              sender_id: localUserGUID,
              username:
                userInfo.personname.LastName +
                ", " +
                userInfo.personname.GivenName,
              date: moment().format("DD MMMM"),
              timestamp: moment().format("HH:mm"),
              system: false,
              saved: true,
              distributed: true,
              seen: true,
              disable_actions: false,
              disable_reactions: false,
              file: file,
              reactions: {
                wink: [],
                laughing: [],
              },
            };

             if (file.audio) {
                 newMessage.file = file;
            }

            messages.push(newMessage);

            this.messages = messages;
}

But the result I get is the following:

I know that if you are recording because the localUrl of the file when opening in another tab works and I listen to what I recorded, the file that is generated is the following:

What needs to be done to make it work.

I have to add the url attribute to the file right?


Remove file APIs

In favour of https://github.com/vapor/core/pull/83

I think this PR is now a breaking PR. So it should probably be closed down since we're in RC phase now.
Besides that, I think the DirectoryConfig should reside in the Files module that I was setting up


Print original path name on error

Signed-off-by: Kenfe-Mickael Laventure<EMAIL_ADDRESS>--
Should make for easier debugging

@mlaventure Have to update handle_nolinux.go as well.

@tonistiigi fn is already the original path in that case

i added the path to the second error message though


feat(opentelemetry): support variable resource attributes

Summary
This change allows render resource attributes as lua code.
For example,
  config:
    traces_endpoint: "https://otlp-red-saas.instana.io:4318/v1/traces"
    resource_attributes:
      host.id: "$(headers.host)"

host.id is <IP_ADDRESS>:9000 or depends on headers.host value.
Checklist

[x] The Pull Request has tests
[x] A changelog file has been created under changelog/unreleased/kong or skip-changelog label added on PR if changelog is unnecessary. README.md
[ ] There is a user-facing docs PR against https://github.com/Kong/docs.konghq.com - PUT DOCS PR HERE

Issue reference

Fix #12538

failing test is expected, let's rebase after https://github.com/Kong/kong/pull/13877 is merged

yes @SuzyWangIBMer I'm trying to get the second approval, thank you for your patience

@samugi  Sorry, other priorities came in. Just updated commits. Thanks for the help all along!


ES-1801: Updating Kafka version as Confluent supports Kafka 3.6

Description
Updating confluentinc/cp-kafka to 7.6.1 in kafka-test.yml and kafka-stateful.yml

Do you have any thoughts on whether the chart version should move to 3.0.1 with this change?
@davidcurrie  / @jacob-scott3141


Fix URL comparison within test context.

Was not a Tock bug. Problem occurred because of environment variables misconfiguration.


Adds heartBeat animation demo

The heartBeat demo is not present on the Animate.css website, this PR adds it.

@eltonmesquita please review this thanks!


Detach perfectscrollbar from pm list

This PR plugs a DOM node leak where removed PM div and children would never get garbage collected due to a live perfectScrollbar JS object being attached and never removed.
Testing Plan:
Deployed in a testing environment over 24 hours and maintained DOM Node counts across patched and unpatched instances
GIFs or Screenshots:

@andersk can you review this?

This only applies to the 2.0.x maintenance branch. master/2.1.x no longer uses perfect-scrollbar per #11792


[LINUX]Exception when downloading a map for live replay ends in a 404

A stack trace caught when trying to watch a live replay:
java.util.concurrent.CompletionException: java.lang.NullPointerException
	<snip>
	at java.base/java.lang.Thread.run(Thread.java:844)
Caused by: java.lang.NullPointerException
	at java.base/java.nio.file.Files.provider(Files.java:99)
	at java.base/java.nio.file.Files.isDirectory(Files.java:2220)
	at com.faforever.client.map.MapService.readMap(MapService.java:223)
	at com.faforever.client.map.MapService.addSkirmishMap(MapService.java:210)
	at com.faforever.client.map.MapService.lambda$downloadAndInstallMap$15(MapService.java:439)
	at com.github.nocatch.NoCatch.noCatch(NoCatch.java:49)
	at com.github.nocatch.NoCatch.noCatch(NoCatch.java:34)
	at com.faforever.client.map.MapService.lambda$downloadAndInstallMap$16(MapService.java:439)
	at java.base/java.util.concurrent.CompletableFuture$UniAccept.tryFire(CompletableFuture.java:714)
	... 22 more

From what I can tell, getPathForMap can return null, but it isn't handled anywhere, so it eventually gets passed to isDirectory.
Looking deeper, I suppose that DownloadMapTask does not download the map correctly, and yet it throws no exception:
Downloading map dualgap_fix_adaptive.v0007 from http://content.test.faforever.com/faf/vault/maps/dualgap_fix_adaptive.v0007.zip
This address returns 404. I think that the DownloadMapTask should throw an IOException in that case, but I'm a bit lost in the task / future callback soup so I'm not sure. Either way, the null from getPathFromMap should be handled, and that I suppose is a separate issue.

This a a known Linux issue i belive. Right you were on Linux and I think that happens in gerneral when downloading maps on linux.

@axel1200  downloading maps works fine on linux if you simlink maps (and mods) folders.
what is an issue though is this :

after clicking ok the game starts up as if started from steam. (intro video and ends up at main menu)

I was told that happens if 5 min did not yet pass

try again 1 min later

There it return null, we should rather make a proper error message there

It is because linux is case sensitive

linux is indeed case sensitive.

IIRC many maps freely mix case between zips, directory names and filenames. Copious use of lowercasing should fix this.

new server new luck


Split legs file by year, added airports, focus on main airports

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 72.54%. Comparing base (7d1127f) to head (b372983).

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@           Coverage Diff           @@
##             main      #49   +/-   ##
=======================================
  Coverage   72.54%   72.54%
=======================================
  Files          16       16
  Lines         987      987
=======================================
  Hits          716      716
  Misses        271      271

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Bug in multiclassification

Issue Description
I think i found bug in multiclassification when i load csv data by DataSetIterator.
Bug appears when classification label index is set as 0.
version: 1.0.0-beta6
code:
DataSetIterator it = new RecordReaderDataSetIterator.Builder(recordReader, someSize)
.classification(0, 3).build();
MultiLayerConfiguration conf = new NeuralNetConfiguration.Builder()
.seed(seed)
.activation(Activation.TANH)
.weightInit(WeightInit.XAVIER)
.updater(new Sgd(0.1))
.l2(1e-4)
.list()
.layer(new DenseLayer.Builder().nIn(numInputs).nOut(3)
.build())
.layer(new DenseLayer.Builder().nIn(3).nOut(3)
.build())
.layer( new OutputLayer.Builder(LossFunctions.LossFunction.NEGATIVELOGLIKELIHOOD)
.activation(Activation.SOFTMAX)
.nIn(3).nOut(outputNum).build())
.build();
result:
Warning: 1 class was never predicted by the model and was excluded from average precision
Classes excluded from average precision: [0]
Warning: 2 classes were never predicted by the model and were excluded from average recall
Classes excluded from average recall: [0, 2]
=========================Confusion Matrix=========================
0 1 2
0 0 0 | 0 = 0
0 1 1 | 1 = 1
0 0 0 | 2 = 2
Conclusion:
When label index is set as 0, confusion matrix always set all values at first column as 0.
If i set label column as last is working fine.

Can you provide the data for us to review?

When label index is set as 0, confusion matrix always set all values at first column as 0.
If i set label column as last is working fine.

Which label index value is correct depends on the actual data layout. If your labels are actually the last column in your CSV, then the only valid configuration for label index is the last column. Set to anything else and it won't/can't work.

Hey, here is example od data where first column is label and the rest ot them are features.
input_2.txt


Lack of examples : VS Code documentation

when i read the VS Code API documentation, i realized that it misses examples explaining how to use the methods, therefore, i think that is complicated for the programmer to utilize those methods in order to create new extensions of vs code.

Did you see this: https://github.com/Microsoft/vscode-extension-samples


Bug to execute az apim

This is autogenerated. Please review and update as needed.
Describe the bug
Command Name
az apim
Errors:
az apim: error: the following arguments are required: _subcommand

To Reproduce:
Steps to reproduce the behavior. Note that argument values have been redacted, as they may contain sensitive information.

Put any pre-requisite steps here...
az apim

Expected Behavior
Environment Summary
Windows-10-10.0.19041-SP0
Python 3.6.6

azure-cli 2.1.0 *

Extensions:
interactive 0.4.3
webapp 0.2.24

Additional Context

apim

Please update to the latest CLI version and see if the issue persists.


Improve treatment of escaped ']' in definition lists

This fixes #546.

@harpocrates I think we have an open issue regarding the hyperlinker:
Haddock.Backends.Hyperlinker.Parser

  parse

    is total

    retains file layout FAILED [1]

    when parsing single-line comments

      should ignore content until the end of line

      should allow endline escaping

    when parsing multi-line comments

      should support nested comments

      should distinguish compiler pragma

    should recognize preprocessor directives

    should distinguish basic language constructs

    should parse do-notation syntax

Failures:

  test/Haddock/Backends/Hyperlinker/ParserSpec.hs:48:

  1) Haddock.Backends.Hyperlinker.Parser.parse retains file layout

       Falsifiable (after 35 tests and 2 shrinks):

       NoTabs " \n"

Randomized with seed 563507825

@alexbiehl I'll take a look. The issue here is amost certainly related to GHC replacing certain character sequences with others.


Committing changes I had to make to get puma working on production.

coverage: 94.41%. remained the same
when pulling 644f3129b474ed1823b7b755a036c54357763164 on fix-puma
into 219b2e3f6c7efff8615a2306d66f4994a4f35f26 on main.


gmrender-resurrect: init

Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option build-use-sandbox in nix.conf on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[ ] Fits CONTRIBUTING.md.

Apparently, it is not compatible with newer libupnp: https://github.com/hzeller/gmrender-resurrect/issues/148


More context to what "failed" means / how to fix it?

Hi there, I've tried to setup Webmentions on my website using webmention.io and brid.gy but I seem to be having some issues with getting tweets backfed.
As you can see in this image, it says failed:

Is there anyway we could get some more context or explanation as to why it has failed? I'm trying to fix things but I can't for the life of me figure out what's wrong.
Any help is much appreciated 🙏🏻

Hey, sorry for the trouble! Looks like your web site originally had https://webmention.io/https://ellyloel.com/webmention as your webmention endpoint, which wasn't right. Looks like you've fixed it since then. I clicked the 🔁 button to retry the webmention, and it worked.

Ah! I tried that too but I must have needed to wait a bit before doing it, thank you for the help and sorry for the trouble 😅


Light angle is returning errors when attempting to apply with active effects

When trying to apply a light emission angle via this module on DAE (without Midi as I know it is currently broken as per the READ ME) it returns an error, informing me that light emission angle must be between 1 and 360 degrees.
The attempted value is 60, but I also tried 180 and 360. The same errors were returned.
This likely will not help, but the console's error, it seemed rather generic but here it is just in case:
Uncaught (in promise) Error: Light emission angle must be between 1 and 360 degrees.     at Token._validateField (opt/foundryvtt/dist/database/odm/abstract.js:1)     at Token.validate (opt/foundryvtt/dist/database/odm/abstract.js:1)     at Function._onUpdate (opt/foundryvtt/dist/database/odm/embedded.js:1)     at async Socket.handleEvent (opt/foundryvtt/dist/sockets.js:1)

Can you show the details of the effect you were trying to apply

Can you show the details of the effect you were trying to apply

Yeah, sorry, here you go. I have tried various things, making fresh items, doing it on different items, but nonetheless, the error appears.

I cant seem to replicate this, is there a chance I can join your server to check

this does not work neither. But if I put the light angle into a new (own) preset it works just fine

Can you send me the actor/item you are using for these tests
also just a point, you need to use UK/US . instead of , for seperating decimals

Can you send me the actor/item you are using for these tests
fvtt-Actor-Whitford_Lowmane (4).zip
also just a point, you need to use UK/US . instead of , for seperating decimals
no, that seems to be a localisation thing. Even if I enter 0.4 and hit save, it shows up as 0,4 the next time I open it up. (Windows here is set to German locals). So both seems to work here.

no, that seems to be a localisation thing. Even if I enter 0.4 and hit save, it shows up as 0,4 the next time I open it up. (Windows here is set to German locals). So both seems to work here.
Oh interesting

You should be using override not add for lightAngle properties.
I cannot replicate this on 0.2.08 with only ATL enabled


allow use of command line arguments again

Commit 245f1e3 breaks the described usage of the clustercheck script while allowing variables to be set in /etc/sysconfig/clustercheck.
This PR still reads values from this file but overwrites them with values given via command line arguments (if given).
Should also fix #20 and #21.

Can I ask how the defaults_extra_file parameter is used?  It looks like all the parameters are positional, so I can only specify it if I've specified all other parameters, including username and password... but the main reason to use a file would be to avoid putting the username and password on the command line


Dilato: Core: Execute the commend line "apt-get install fcitx" and then reboot, the Brackets will be deleted

Test build: 1.12.0-17538
Test environment: Debian 9.3.0

Install Brackets
Open termian
Input "su" then input the root password to enter into the root mode
Input "apt-get install fcitx" then press Enter key
Input "reboot" --- The system will be reboot
Check the Brackets

Result: The Brackets doesn't in the system
Expect: The Brackets shoud not be reomved

This issue only happened in the first time user execude the commend line (Step 4 and 5)

@ZuoBaogang Is this happening with only 1.12 and not with with 1.11 Linux version?

Hi @nethip  , I just checked this issue with 1.11
This issue also reproduce
Test enveronment: Debian 9.3.0 64bit
Test build: 1.11.0-17524 download from: https://github.com/adobe/brackets/releases

@ZuoBaogang thanks for updating.


Work Package wp-ho.csv

I would like to work on this please!

Hi,
I was assigned this package as well.
Please find the reuse document attached in the comment.
Thanks and Regards.
Moksh Jain
wp-ho-results.csv

wp-ho-results.csv
Results of Phase 1

Results of Phase 2:
wp-ho-results.csv

Results of Phase 2:
wp-ho-results.csv
2:


Download update in background

Resolved / Related Issues
Items resolved / related issues by this PR.

Closes #issue...

Details of Changes
Add details of changes here.

Changed update service to download available update in the background before showing the Update button. When the update has finished downloading the button shows to install the update.
MSIX bundle is downloaded to the application folder.

Validation
How did you test these changes?

[x] Built and ran the app

Would it be better to use Path.GetTempPath for the download location instead of the app folder?

@puppetsw does this work for the store version as well?

@puppetsw does this work for the store version as well?

No, only the sideload version at the moment.

MSIX bundle is downloaded to the application folder.

How did this work before?

MSIX bundle is downloaded to the application folder.

How did this work before?

The package manager downloaded it to whereever it does internally using the appinstaller. This PR takes that away and we handle it manually, so we can do it in the background.

What happens if the app is closed before the download finishes?

What happens if the app is closed before the download finishes?

At the moment it is just terminated and starts again (not resume) at startup.


zb_config_platform.h missing ifdef cplusplus extern C (TZ-1405)

Answers checklist.

[X] I have read the documentation ESP Zigbee SDK Programming Guide and tried the debugging tips, the issue is not addressed there.
[X] I have updated ESP Zigbee libs (esp-zboss-lib and esp-zigbee-lib) to the latest version, with corresponding IDF version, and checked that the issue is present there.
[X] I have searched the issue tracker for a similar issue and not found a similar issue.

IDF version.
5.1.5
esp-zigbee-lib version.
1.6.0
esp-zboss-lib version.
1.6.0
Espressif SoC revision.
ESP32C6
What is the expected behavior?
All functions in header should be inside
What is the actual behavior?
#ifdef __cplusplus
extern "C" {}
#endif
Steps to reproduce.
include file in cpp project
More Information.
No response

Could you please add extern "C" for this header when using it in your project? Alternatively, use esp_zigbee_core.h instead of zb_config_platform.h.

Done that and reported the issue

The cplusplus extern C is missing in most ZBOSS header files, and we don't expect our users to include ZBOSS header files in their applications, but should use the esp-zigbee APIs instead.


Contributing.md beatrix droid

Summary
Details and comments

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

updated the contribution.md file as requested, and created another markdown where the guide on git is, and linked it to the contribution,md file. This makes the contribution.md file less cluttered, as that guide is now gone, but at the same time, users that will need the guidance to make a PR will be pointed to the right resources.

i have closed this pull request and opened another one with the changes that were requested

Linking new PR here https://github.com/qiskit-community/ecosystem/pull/182


Game Tutorial Triggers Implementation

Triggers Implementation
Task: Game Tutorial Triggers Implementation.
Feature: Game Tutorial
The aim of this ticket is to implement triggers within the game that will determine when and how the tutorial is displayed to players.
Milestones
List of steps that need to be completed for this task.

[x] Goal 1:Trigger Design and Integration with Game Logic
[ ] Goal 2: Testing and Documentation
[ ] Goal 3: Feedback and optimization

Member
@khushijain0302

Delayed until next sprint

Scrapped for this sprint


Support InClusterConfig when running ks

Is this a BUG REPORT or FEATURE REQUEST?: Bug
/kind bug
What happened: Ran ks from a pod in the cluster and it didn't pick up the standard service account stored in /var/run/secrets/kubernetes.io/serviceaccount/
What you expected to happen: I expected it to automatically pick up the service account associated with the pod.
How to reproduce it (as minimally and precisely as possible):  Run a pod in a kubernetes cluster and run kubectl version (see that it properly communicates to the server) and then run ks validate default (or anything to cause ks to talk to kube-apiserver).
Anything else we need to know?: I hit this while working on deploying our ksonnet resources in a CICD pipeline.
Environment:

ksonnet version (use ks version):

ksonnet version: dev-2018-05-23T18:51:14+0000
jsonnet version: v0.10.0
client-go version:

(built using go get ... + make install at the HEAD of master)

Kubernetes version (use kubectl version):

Client Version: version.Info{Major:"1", Minor:"10", GitVersion:"v1.10.2", GitCommit:"81753b10df112992bf51bbc2c2f85208aad78335", GitTreeState:"clean", BuildDate:"2018-04-27T09:22:21Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"10", GitVersion:"v1.10.1", GitCommit:"d4ab47518836c750f9949b9e0d387f20fb92260b", GitTreeState:"clean", BuildDate:"2018-04-12T14:14:26Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"linux/amd64"}

Unfortunately, I'm not very familiar with the client-go API to know the proper convention, but I found where ksonnet initializes the client InitClient in pkg/client/client.go.  I believe you need to call InClusterConfig in order to initialize this specific case, but I'm not clear on the proper way to do this (otherwise, I'd happily submit a PR).
If you give me some guidance, I'd be happy to submit a PR.  I think its probably pretty easy to fix though.  So whatever works 😄

Throwing my 2 cents here...
Seems like direct support is not going to be added, but here is some workaround:
https://github.com/ksonnet/ksonnet/issues/431
using
ks env add kkk8s --server=https://${KUBERNETES_SERVICE_HOST}:${KUBERNETES_SERVICE_PORT_HTTPS} --token $(cat /var/run/secrets/kubernetes.io/serviceaccount/token)

Anyways, kudos to anyone implementing client-go in cluster config detection.


bump iojs up to @2.x

iojs have already released 2.2.1 version, so node-html-to-text need to update engine field.
btw, do you really need this field? cause it’s not only complaining and not really care about engine versions, as far as library is not testing under all stated engines in the CI

it is out of scope for this issue, but do you want me to pull request travis config?

Thanks for the pull request

pls, publish patch version =)

On the way.


Convert urls to links

Urls are converted to links: {urL}

Sorry for the delay and thanks for the PR. If you're still interested in having this feature, I'll be happy to merge it if you can refactor it to use https://github.com/sindresorhus/linkify-urls instead.

I'm afraid I can't since I've already deleted my fork. I haven't used hotel in a while so I'm no interested in this PR anymore, I'll close this PR.
Thanks again for this great app.

No worries, thanks for the feature idea 👍


[p2p] Store DHT state in SphereFS

We should store DHT state, like recent peers, in SphereFS, rather than the out-of-the-box MemoryStore

Linking to original feedback for additional context: https://github.com/subconsciousnetwork/noosphere/pull/93#discussion_r999934235


i can add restore / delete task functionality  along with popup messages !!

Can you assign me to this?

This awaits #85 to be complete.


desktop games import script

Made a separate import script because Desktop Game Sites expects different things from mass e.me sites. Such as promo images, desktop support, no assumptions on regions or popularity, featured-game tags and categories.

[x] Need to mock the image fetching code.
[x] More tests for image saving code.

r? @diox

Note: you'll need to remember to run a reindex after this is done, since you are not triggering any tasks that does that for you.

Note: in that CSV, the images on dropbox fail to load for me (410 error)

Yeah, I need to reupload the images somewhere else where the CDN doesn't expire the links.

Thanks, I was meaning to add a reindex for after images are saved.

thanks!


MODELIX-584 Update roadmap

Still needs to be tested and we will probably have to adjust the dates

has to be merged after https://github.com/modelix/modelix.docs/pull/27


[hotfix][docs] Readme review to clarify heading formatting

I clarified how to format a heading casing wise.

I'm closing this as "Abandoned", since there is no more activity and the code base has moved on quite a bit. Please re-open this if you feel otherwise and work should continue.


Don't ignore src/ directory for bower installs

The src/ files are necessary for source maps to work.
This pull request is based on https://github.com/web-animations/web-animations-js/pull/76 using the newest dev branch instead of master.

Hi, I'm wondering if a new 2.2.x release could be done with this fix please?
I have this error
[Error] Failed to load resource: the server responded with a status of 404 (Not Found) (web-animations-next-lite.min.js.map, line 0)

@alancutter


Contribution guidelines / wiki improvements

Description
As a freelance working with a company considering using this dashboard, I am looking for building the package locally. I might contribute by pushing some PR on this repo.
I dug into the documentation : https://dashboard.flowfuse.com/contributing/ and I think the Installing Locally section is not complete enough.
I assume simply cloning without any npm linking will not register nodes. Can you share your workflow for upgrading this section for a complete setup ? (npm link or yarn link or even yalc for dependancies).
Epic/Story
No response
Have you provided an initial effort estimate for this issue?
I have provided an initial effort estimate

Thanks for raising the issue @IfnotFr
We've actually just had a community contributor upgrade them considerably: https://github.com/flowforge/flowforge-nr-dashboard/pull/205
I've also just refined these slightly in this PR too: https://github.com/flowforge/flowforge-nr-dashboard/pull/207
Please let me know if these suffice, and if so, feel free to close out this issue. Any issues, let us know, and we can refine further.

Hi @IfnotFr - I've just published the v0.6.0 (Release Notes) which includes full support for third-party widget integration.
I've also made big improvements to the documentation for setting up locally here and making third party widgets here

Please let me know if this is satisfactory, and then we can close this issue out. Look forward to seeing what you produce

Awesome, it looks very promising ! Can't wait to try it :) Thanks for your reactivity


klone has been moved to the cert-manager repo

Update the url of the klone tool.

Self-approving this PR, since this is not yet in-use.

/approve
/lgtm


[WIP][DRAFT] SSL Certs

Vercel https://vercel.com/docs/projects/domains/working-with-ssl
Netlify https://docs.netlify.com/domains-https/https-ssl/#netlify-managed-certificates

Superseded by https://github.com/doseiai/engine/pull/64


Liquibase Distribution distinction between OSS vs commerical

Liquibase Version: 4.12.0
Database Vendor & Version: ALL
Description
Liquibase product contains both the commercial components as well as the OSS components. The commercial components contain the prohibited AGPL License through some scanners.
Would there be any choice to have two different release.?
liquibase-oss-4.12.0.tar.gz
liquibase-cc-4.12.0.tar.gz

we are in the process of making this happen... stay tuned!

going to close this issue and will use https://github.com/liquibase/liquibase/issues/2629 to represent the work; will make sure to use that issue to notify back when all the splits are done and available


Subclassing: gtk_container_forall () is missing

I'm currently trying to port this widget from Vala to Rust:
https://gitlab.gnome.org/GNOME/gnome-games/-/blob/master/src/ui/titlebar-box.vala#L254
The current blocker is the missing gtk_container_forall func:
https://developer.gnome.org/gtk3/stable/GtkContainer.html#gtk-container-forall
As I understand this func is really necessary to create own custom widgets.
Related PR: https://github.com/gtk-rs/gtk/pull/1035

Done in 9e50ccaebee21523afe23ab1ef83842af4ae5a39

@sdroege Sorry if this is the wrong location to ask (which it likely is, but I have no clue where to ask...). When can a version which contains the gtk_container_forall be expected on crates.io? I would really need this patch and currently use the git repo as dependency, but that does not feel legit to include into my project's main branch as it is no stable version.

@sdroege Sorry if this is the wrong location to ask (which it likely is, but I have no clue where to ask...). When can a version which contains the gtk_container_forall be expected on crates.io? I would really need this patch and currently use the git repo as dependency, but that does not feel legit to include into my project's main branch as it is no stable version.

In a couple of weeks hopefully. There are a few more things waiting to be finished first.

In a couple of weeks hopefully. There are a few more things waiting to be finished first.


[window-state] JS api should await promise in saveWindowState

The main use case for this API, I assume, is to call saveWindowState before closing the window via appWindow.close.
In it's current state this does not work because the window is closed before the state is actually updated and saved, on my hardware at least.
The other functions in the API should probably be awaited as well.
On a side note exposing the window.update_state function through the API would allow the user to mimic what the plugin does on the onCloseRequested event, and prevent saving all window states when one window is closed.
Again this invoke call would have to be awaited in this case to work correctly.
After looking more at the plugin this would require a bit of a refactor.
This whole issue still stems from the fact that appWindow.close does not invoke the onCloseRequested event on the window. (tauri-apps/tauri#5288)

Thanks for the report :)

On a side note exposing the window.update_state function through the API would allow the user to mimic what the plugin does on the onCloseRequested event, and prevent saving all window states when one window is closed.

How so (genuine question)? From what i can see, what the plugin does on close is to call save_window_state (which uses update_state internally) which is what the saveWindowState function does too.
Did you test if adding await actually fixes the issues you're having? Because from the usage of this function i've seen so far it seemed to work as-is (we should add the await anyway)

How so (genuine question)? From what i can see, what the plugin does on close is to call save_window_state (which uses update_state internally) which is what the saveWindowState function does too.

It's a bit of a premature optimization, but in my case it causes 2 disk writes, one when you call the saveWindowState and another one when the plugins Exit event handler fires.

Did you test if adding await actually fixes the issues you're having? Because from the usage of this function i've seen so far it seemed to work as-is (we should add the await anyway)

Yes, albeit I didn't modify the plugin, but I mimicked what it does whilst adding an await.


Question about NumberFormat: '10,00' parsed as '1000.0'

var nf = new NumberFormat("#.00");
print(nf.parse('10,00'));

1000.0

Why is result 1000.0? I do not have ',' comma in format...
Is it possible to get error for parsing this number?

We ignore the grouping separators when parsing, as well as spaces and non-breaking spaces. If you're trying to parse precisely a format you could verify that printing the result is precisely equal to the input string. You could also create a locale with no grouping separator and use that.


rubocop: 1.10.0 -> 1.11.0

Motivation for this change
https://github.com/rubocop/rubocop/releases/tag/v1.11.0
Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[x] NixOS
[x] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

This is a semi-automatic executed nixpkgs-review with nixpkgs-review-checks extension. It is checked by a human on a best effort basis and does not build all packages (e.g. lumo, tensorflow or pytorch).
If you have any questions or problems please reach out to SuperSandro2000 on IRC.
Result of nixpkgs-review pr 115269 run on x86_64-darwin 1

1 package built:

rubocop


chore(button): updated outline button color to look different than plain text

"Ghost buttons" btn-outline-primary border-0 are often use to get an affordance of clicking, with hover effects and all, without having a full button style. This change updates the color for primary to remain blue, so it behaves like a clickable hyperlink and not plain text.

Closing, cherry-picked this commit to a new branch to clean things up.


Column name decoding issue after v4.0.3

I'm connecting to a Teradata database from Python 3.6 on macOS. I'm seeing strange column names when I use v4.0.5 and v4.0.6 of pyodbc that I wasn't seeing before with v4.0.3. It almost looks like a decoding issue.
In v4.0.3, I can do this:
connection = pyodbc.connect(dsn='mydsn')

# Encoding/decoding for Teradata+Python3.6 as directed in the docs
connection.setdecoding(pyodbc.SQL_CHAR, encoding='utf-8')
connection.setdecoding(pyodbc.SQL_WCHAR, encoding='utf-8')
connection.setencoding(encoding='utf-8')

cursor = connection.cursor()
cursor.execute("SELECT TOP 5 * FROM mydb.mytable")
cursor.description

Which results in,
(('STATE', str, None, 2, 2, 0, True),
 ('ZIP', str, None, 5, 5, 0, True),
 ('CITY', str, None, 30, 30, 0, True),
 ('ADDRESS', str, None, 30, 30, 0, True))

However, if I repeat the same exact process in v4.0.5 and v.4.0.6, I get this:
(('S\x00T\x00A', str, None, 2, 2, 0, True),
 ('Z\x00I', str, None, 5, 5, 0, True),
 ('C\x00I\x00', str, None, 30, 30, 0, True),
 ('A\x00D\x00D\x00R', str, None, 30, 30, 0, True))

Have I set up my connection incorrectly, or do my findings point to a breaking change after v4.0.3?
Thanks.

What I found out after 4.0.3 is that different databases return the column names differently.  SQL Server, PostgreSQL, and MySQL all return them as UTF-16LE regardless of the data encoding.
I've added a 3rd decoding option for this but defaulted it to utf-16le.  Please add this:
connection.setdecoding(pyodbc.SQL_WMETADATA, encoding='utf-8')

Sorry for the trouble, but each database does things different instead of following the spec.  (In this case the spec says it should be UTF16LE.).
I'm considering creating a "factory" that can be configured and then connections created using factory.connect().  I'd also consider adding something like:
factory.autoconfigure('teradata')

with setting people have tested.  (I don't have teradata.)
In the meantime I'll try to keep the docs up to date.  Please let me know if this works for you.

So here is my updated configuration block,
connection.setdecoding(pyodbc.SQL_CHAR, encoding='utf-8')
connection.setdecoding(pyodbc.SQL_WCHAR, encoding='utf-8')
connection.setdecoding(pyodbc.SQL_WMETADATA, encoding='utf-8')
connection.setencoding(encoding='utf-8')

And here is the new output from cursor.description in my example:
(('S\x00\x00\x00T\x00\x00\x00A\x00', str, None, 2, 2, 0, True),
 ('Z\x00\x00\x00I\x00', str, None, 5, 5, 0, True),
 ('C\x00\x00\x00I\x00\x00\x00', str, None, 30, 30, 0, True),
 ('A\x00\x00\x00D\x00\x00\x00D\x00\x00\x00R\x00', str, None, 30, 30, 0, True))

So the extra decoding setting did have an effect, but it's still not right.
Any ideas? If I knew more about encoding/decoding, I might be able to help troubleshoot this better, sorry.

Not sure if this is helpful at all: http://info.teradata.com/HTMLPubs/DB_TTU_15_10/index.html#page/Connectivity/B035_2509_035K/2509ch08.10.04.html
I don't fully understand it, but the section "When Making SQLColAttribute Calls" feels like it might be relevant.

Dang - the fact that drivers don't follow specs means I have to support every possible combination of buffers everywhere!  I'm calling SQLDescribeColumnW which only accepts a SQLWCHAR buffer.  Can you try one more thing?  It shouldn't be necessary so I'm not expecting a change but I'd like to get it tested:
connection.setdecoding(pyodbc.SQL_WMETADATA, encoding='utf-8', ctype=pyodbc.SQL_CHAR)

Please note that this could cause a crash if it is what I think - it might look right but actually use bytes beyond the those in the buffer.
I'll try to find a teradata database I can test with soon.  Do you know of any public ways to do so?  I thought the only test version they offered was a full VM for something I don't use (I use VirtualBox).

The extra ctype keyword didn't have any effect on the description output. No crash, either.
It looks like there's some documentation on character sets, but again, unfortunately I don't really understand how all the pieces fit together: http://info.teradata.com/HTMLPubs/DB_TTU_15_10/index.html#page/Connectivity/B035_2509_035K/2509ch08.10.32.html

Oh, and I use my company's instance of Teradata and I'm not aware of a public version.

Is it worth looking into how the teradata package does things?

I'm going to have to find a way to test it so I can dump out buffers raw.  In the meantime you can try these and see if they help:
connection.setdecoding(pyodbc.SQL_WMETADATA, encoding='utf-16le')
connection.setdecoding(pyodbc.SQL_WMETADATA, encoding='utf-16be')

The first is what it looks like should be correct, but is also the default. Maybe there is a bug and the default is getting overwritten?

The first gives me the same output as in my original post (as expected, I think). The second gives me asian characters of some kind :)
I'll keep playing around with different encoding/decoding options on my own, including the CharacterSet connection option for the Teradata driver.

OK - in the meantime I'm downloading Teradata Express so I can test in a VM.  (I forgot you can run a VMWare vmdk in VirtualBox.). However, I'm wondering about the driver - it looks like it uses iODBC.  Are you using a standard binary wheel of pyodbc or building your own with iODBC?

I built pyodbc myself against my driver manager, which is iODBC from Homebrew.

There might be a SQLWCHAR size difference then.  You might try "utf32le" and "utf32be".  It'll take me a bit longer to setup with iODBC.  Working on it.

TL;DR - Use 4.0.7 and set the decoding to:
cnxn.setdecoding(pyodbc.SQL_WMETADATA, encoding='utf-32le')

Whew ... it took forever to get to the point where I can test.   I don't know if this is buggy behavior in the Teradata driver (my guess) or if iODBC returns 4-byte SQLCHAR for metadata.  Either way I've added a hack that if the metadata is set to one of the UTF-32 encodings it assumes sizeof(SQLWCHAR) is 4.
In case you are wondering how all of this started and why we can't use what seemed to work before, I switched to the Unicode version of SQLDescribeCol to handle Unicode column names.  That's not something I wanted to / could revert.

Thanks for fixing this, it works perfectly now! The amount of work you put into this project is heroic.

Hi,
I met a  problem when I connected to a database(product of my company) with pyodbc4.0.17 and unixODBC2.3.4 that the column  name can't be decoded.Then I found that the unixODBC selects whether to call an API with W according to the application type(ANSI or Unicode) but  pyodbc calls the  SQLDescribeColumnW by default instead of according to the application type so that our driver can not handle inconsistent coding requirements before and after.I want to know that whether the design will result in only support for unicode? @mkleehammer

Just adding another data point- I saw the same issue connecting to Vertica, so at the least it's not unique to Teradata. (py3.6, macOS, pyodbc 4.0.22)
As described above, setting the metadata decoding to utf-32le worked for me:
conn_vert.setdecoding(pyodbc.SQL_WCHAR,` encoding='utf-8')
conn_vert.setdecoding(pyodbc.SQL_CHAR, encoding='utf-8')
conn_vert.setdecoding(pyodbc.SQL_WMETADATA, encoding='utf-32le')
conn_vert.setencoding(encoding='utf-8')

conn_vert.setdecoding(pyodbc.SQL_WMETADATA, encoding='utf-32le')
was a winner for me also....

Everyone, comment out, or delete, the following line of code line 80 in hyper_file_generator.py in the library  (if you're creating a hyper):
print((col[0].decode('utf-8')))
This immediately fixed the issue for me.


DTH logging messages not handled

I'm noticing what I think are messing from the DTH coming through into the console output but can't immediately see how to control/filter them as per other messages. Any thoughts/guidance?
For examples;
Client accepted <IP_ADDRESS>:53398
Sample output;
{"Event":"log","Body":{"LogLevel":"INFORMATION","Name":"OmniSharp.Startup","Message":"Omnisharp server running using stdio at location '/users/ibattersby/.spacemacs.layers/my-csharp/extensions/omnisharp-emacs/test/MinimalSolution/' on host -1."},"Seq":1,"Type":"event"}
{"Event":"log","Body":{"LogLevel":"INFORMATION","Name":"OmniSharp.Dnx.DnxPaths","Message":"Using runtime '/Users/ibattersby/.dnx/runtimes/dnx-mono.1.0.0-beta4'."},"Seq":2,"Type":"event"}
{"Event":"log","Body":{"LogLevel":"INFORMATION","Name":"OmniSharp.Dnx.DnxProjectSystem","Message":"Scanning '/users/ibattersby/.spacemacs.layers/my-csharp/extensions/omnisharp-emacs/test/MinimalSolution/' for DNX projects"},"Seq":3,"Type":"event"}
{"Event":"log","Body":{"LogLevel":"INFORMATION","Name":"OmniSharp.Dnx.DnxProjectSystem","Message":"Found project '/users/ibattersby/.spacemacs.layers/my-csharp/extensions/omnisharp-emacs/test/MinimalSolution/minimal/project.json'."},"Seq":4,"Type":"event"}
{"Event":"log","Body":{"LogLevel":"VERBOSE","Name":"OmniSharp.Dnx.DesignTimeHostManager","Message":"/Users/ibattersby/.dnx/runtimes/dnx-mono.1.0.0-beta4/bin/dnx \"/Users/ibattersby/.dnx/runtimes/dnx-mono.1.0.0-beta4/bin/lib/Microsoft.Framework.DesignTimeHost/Microsoft.Framework.DesignTimeHost.dll\" 53398 45870 9afc9035-9d2e-468d-b9e3-da3a143d02c1"},"Seq":5,"Type":"event"}
Listening on port 53398
{"Event":"log","Body":{"LogLevel":"INFORMATION","Name":"OmniSharp.Dnx.DesignTimeHostManager","Message":"Running DesignTimeHost on port 53398, with PID 45874"},"Seq":6,"Type":"event"}
{"Event":"log","Body":{"LogLevel":"INFORMATION","Name":"OmniSharp.Dnx.DnxProjectSystem","Message":"Connected"},"Seq":7,"Type":"event"}
{"Event":"log","Body":{"LogLevel":"VERBOSE","Name":"OmniSharp.Dnx.DnxProjectSystem","Message":"[ProcessingQueue]: Start()"},"Seq":8,"Type":"event"}
Client accepted <IP_ADDRESS>:53398
Client accepted <IP_ADDRESS>:53398
{"Event":"log","Body":{"LogLevel":"VERBOSE","Name":"OmniSharp.Dnx.DnxProjectSystem","Message":"[ProcessingQueue]: Post((9afc9035-9d2e-468d-b9e3-da3a143d02c1, Initialize, 1) -> {
  \"Configuration\": null,
  \"ProjectFolder\": \"/users/ibattersby/.spacemacs.layers/my-csharp/extensions/omnisharp-emacs/test/MinimalSolution/minimal\"
})"},"Seq":9,"Type":"event"}
{"Event":"log","Body":{"LogLevel":"VERBOSE","Name":"OmniSharp.Dnx.DnxProjectSystem","Message":"[ProcessingQueue]: Receive ((9afc9035-9d2e-468d-b9e3-da3a143d02c1, ProjectInformation, 1) -> {
  \"Name\": \"minimal\",
  \"Frameworks\": [
    {
      \"FrameworkName\": \"DNX,Version=v4.5.1\",
      \"FriendlyName\": \"DNX 4.5.1\",
      \"ShortName\": \"dnx451\",
      \"RedistListPath\": null
    }
  ],
  \"Configurations\": [
    \"Debug\",
    \"Release\"
  ],
  \"Commands\": {},
  \"ProjectSearchPaths\": [
    \"/users/ibattersby/.spacemacs.layers/my-csharp/extensions/omnisharp-emacs/test/MinimalSolution\"
  ],
  \"GlobalJsonPath\": null
})"},"Seq":10,"Type":"event"}
{"Event":"ProjectChanged","Body":{"DnxProject":{"Path":"/users/ibattersby/.spacemacs.layers/my-csharp/extensions/omnisharp-emacs/test/MinimalSolution/minimal/project.json","Name":"minimal","Commands":{},"Configurations":["Debug","Release"],"ProjectSearchPaths":["/users/ibattersby/.spacemacs.layers/my-csharp/extensions/omnisharp-emacs/test/MinimalSolution"],"Frameworks":[],"GlobalJsonPath":null,"SourceFiles":null}},"Seq":11,"Type":"event"}

The JSON data is always printed as a single line, so it will be save to check for the first character being {

Hi @jrieken. Yes I'm quite aware of that, an ironically some buffering I have in place catch this so I don't get errors in my implementation, however it's something that surely should be happening no? It should come through as a log message.

yeah, I guess when launching the DTH process we shouldn't tell it to re-use in/out put wire that up logger of some sort

OK, I'll try and figure it out if its seemingly possible.

Maybe not: https://github.com/aspnet/dnx/blob/1bb5a1d24fcd7d21d2278528c243e2b9f181c43d/src/Microsoft.Dnx.DesignTimeHost/Program.cs#L79
In an ideal world they'd have provided an abstraction on top of this, or just use ILogger. // cc @davidfowl

I think we can control that here: https://github.com/OmniSharp/omnisharp-roslyn/blob/master/src/OmniSharp.Dnx/DesignTimeHostManager.cs#L69

Good point.